24

ISSA PREEMINENT TRUSTED GLOBAL

INFORMATION SECURITY COMMUNITY

AbstractCommon Criteria (CC) is an international standard (ISO/IEC 15408) for evaluating the security capabilities of IT prod-ucts and systems. Originated from Trusted Computer System Evaluation Criteria, Canadian Trusted Computer Product Evaluation Criteria, and Information Technology Security Evaluation Criteria in 1996, CC has replaced its predecessors with a unified security criterion and has been widely em-braced by almost thirty countries. Since its inception, how-ever, it has also received a lot of criticisms. In this paper, the authors will summarize the shortcomings and limitations of CC, and then offer advice on how to improve it.

Common Criteria tries to provide customers of an IT security product or system with a level of assurance that the product or system performs the security

functionality as specified to counter targeted threats, and that the security functionality is also correctly implemented. Despite its wide use, CC has been constantly receiving criti-cisms from security researchers, IT security product vendors, consumers, and governments. To address those criticisms, CC itself has undergone several changes. But the updates are still not enough to address many arguments on the standard. Hence, CC has to improve to remain relevant to the latest de-velopment in security assurance, and adopt the good prac-tices of security in the development life cycle. In this paper, the authors are going to summarize the challenges to CC and then offer some advice on improving it.

Common Criteria overviewCommon Criteria evaluation provides the consumers of an IT security product or system, i.e., the Target of Evaluation (TOE), with a level of assurance that the product or system

performs the security functionality as specified to counter the targeted threats, and that the security functionality is correctly implemented as well. The framework of an evalua-tion is defined by the Security Target (ST), which may or may not claim conformance to a Protection Profile (PP), which is a general statement of security requirements written for a cat-egory of security products or systems. The ST is a particular statement of provision for the evaluation of a specific TOE, including all information that a PP may contain as well as specifying the security functions and security mechanisms.1

In CC, the security requirements are categorized as security functional requirements (SFR) and security assurance re-quirements (SAR). Basically, the protection profile or secu-rity target describes the SFRs that a TOE tries to meet, and specifies the SARs that the developer of the TOE tries to apply to demonstrate a level of assurance that the TOE meets its SFRs and is correctly implemented.2 3

Basically, CC assurance is achieved by carrying out the fol-lowing TOE evaluation activities:

• Analysis and checking of processes and procedures

• Analysis of guidance documents

• Analysis of the TOE design against the SFRs

• Analysis of functional tests

• Independent functional testing

• Analysis for vulnerabilities

• Penetration testing

1 “Common Criteria for Information Technology Security Evaluation, Part 1: Introduction and general model, Version 3.1 Revision 3, CCMB-2009-07-001.”

2 “Common Criteria for Information Technology Security Evaluation, Part 2: Security functional components, Version 3.1 Revision 3, CCMB-2009-07-002.”

3 “Common Criteria for Information Technology Security Evaluation, Part 3: Security assurance components, Version 3.1 Revision 3, CCMB-2009-07-003”

Common Criteria is an international standard (ISO/IEC 15408) for evaluating the security

capabilities of IT products and systems. In this article, the authors summarize the challenges

to Common Criteria and offer advice on improving it.

By Changying Zhou and Stefano Ramacciotti

Common Criteria:Its Limitations and Advice on Improvement

ISSA Journal | April 2011

©2011 Information Systems Security Association • www.issa.org • editor@issa.org • Permission for author use only.

25

Common Criteria | Changying Zhou and Stefano Ramacciotti

CC pre-defines seven Evaluation Assurance Levels (EALs): EAL1, the lowest level with least rigor of evaluation effort, to the EAL7, the highest level.4 The EAL quantifies how sure you can be that a TOE conforms to the declared specifications and that they are correctly implemented. Common Evalua-tion Methodology (CEM) is mutually recognized among CC community only for EAL1 to EAL4.

There are at least two different approaches to IT security5: avoid threats vs. accept or mitigate risks. It is realized that it is impractical and difficult to achieve complete risk avoidance. In fact, theoretically it is undecided to determine whether a general TOE is secure or there is not any vulnerability in a TOE: such a question is equivalent to the halting problem of a Turing Machine, which is mathematically undecided.6 7

Hence, CC evaluation is not aimed to avoid threats and risks, but to mitigate threats and risks. This is reflected by the attack potential, which is specified at each EAL for the TOE to resist against. In other words, the TOE is exploitable if the attacker obtains an attack potential higher than the designated attack potential at a given EAL.

Although CC has become an international standard of IT security evaluation, it still contains many issues and limi-tations. Some issues are related to evaluation process. Espe-cially, CC is criticized as being costly and time consuming. Meanwhile, there are issues in general evaluation metho-dology. Particularly, its limitation on vulnerability analysis is eminent: CC is not good at addressing security flaws in product implementation. The methodology of vulnerability assessment in CC is too generic, not rigorous to identify vul-nerability in implemetation, and does not take into account vulnerabilities specific to individual technology area. Limita-tions of CC evaluation will be further examined in the subse-quent section, which is followed by advice on how to address part of technical concerns about CC.

Limitations of CC evaluation and certificationCommon Criteria has been receiving criticisms on its short-comings and limitations from many IT security researchers, security practitioners, governments, and consumers/vendors of IT products. Among the already identified and well rec-

4 “Common Criteria for Information Technology Security Evaluation, Part 3: Security assurance components, Version 3.1 Revision 3, CCMB-2009-07-003.”

5 Bruce Schneier, “Risk, Complexity, and Network Security,” April 2001, http://bt.counterpane.com/presentation1.pdf.

6 Matt Bishop, “Computer Security: Art and Science”, Addison-Wesley, December 2002.

7 Brian Chess, Gary McGraw, “Static Analysis for Security”, IEEE Security and Privacy, Volume 2, Issue 6, November 2004.

ognized CC shortcomings and limitations,8 we realize that some are still relevant and need to be addressed for current CC standard (i.e., CC version 3.1). Meanwhile, there are some novel shortcomings identified by the authors. They are elabo-rated as below.

Shortcomings in evaluation processCommon Criteria does not require implementation of many good security practices in the development life cycle in order to improve the product’s security level. Thus, completion of the CC evaluation does not necessarily mean a more secure product.

CC’s evaluation life cycle is also lengthy, basically due to the complexity of the CC terms and methodology that are dif-ficult to understand and follow by the developers. Hence, much resource is spent on preparation of the evaluation. Due to the lengthy process, by the time the work is completed, a new version of the evaluated product may have been rolled out.

Another concern is that the reality of rapidly evolving vulner-ability attack landscape and newly found flaws might make the evaluation outdated due to lengthy certification life cycle.

Figure 1 shows the cost and time spent for the Common Cri-teria evaluation. Although it originates from a 2006 GAO Report,9 the cost and time spent on the TOE evaluation un-der CC Version 3.1 have not significantly changed since then.

Figure 1 – Cost and time spent on evaluation at EAL1 – EAL4

8 “Common Criteria and answering the question 'Is it Safe'“, http://blogs.msdn.com/sdl/archive/2007/12/20/common-criteria-and-answering-the-question-is-it-safe.aspx; “Information Assurance: National Partnership Offers Benefits, but Faces Considerable Challenges”, Report GAO 06-392, United States Government Accountability Office, March 2006; “Common Criteria”, http://en.wikipedia.org/wiki/Common_criteria; Jackson, William, “Under attack: common criteria has loads of critics, but is it getting a bum rap?”, Government Computing News (GCN), Aug 10, 2007; Joab Jackson, “Symantec: Common Criteria is bad for you”, Government Computing News (GCN), May 04, 2007; Rana Aamir Raza Ashfaq, Mohammad Qasim Khan, “Analyzing Common Criteria Shortcomings to Improve its Efficacy”, School of Computing, Blekinge Institute of Technology, Soft Center, SE-37225 RONNEBY, SWEDEN; “Security in the Software Lifecycle”, Department of Homeland Security; “Software Security Assurance – State of the Art Report”, Information Assurance Technology Analysis Center (IATAC) and Data and Analysis Center for Software (DACS).

9 “Information Assurance: National Partnership Offers Benefits, but Faces Considerable Challenges”, Report GAO 06-392, United States Government Accountability Office, March 2006.

Cost: Dollars in thousands350

300

250

200

150

100

50

02 3 4

Evaluation assurance level Evaluation assurance level

Source: GOA analysis of data provided by laboratories.

Time Spent: Months25

20

15

10

5

02 3 4

The TOE is exploitable if the attacker obtains an attack potential higher than the designated attack potential at a given EAL.

ISSA Journal | April 2011

©2011 Information Systems Security Association • www.issa.org • editor@issa.org • Permission for author use only.

26

Common Criteria | Changying Zhou and Stefano Ramacciotti

it is eventually deployed in the real world. A vendor can even restrict the evaluation to certain security features and make certain assumptions about the operating environment and the threats faced by the product in that environment. Such assumptions could be unrealistic for the common use of the product.

Shortcomings of vulnerability assessmentNowadays many security attacks are launched against the vulnerabilities in the implementation of security function-alities (e.g., buffer overflow); thus people are not only con-cerned about the verification of the TOE security functional specification, but are also getting more concerned about whether the TOE does not do what it should not do in order to prevent the compromise of the whole security posture of the system which the TOE is part of. Hence, the vulnerability analysis plays a significant role in achieving the assurance of the TOE’s security.

In CC version 3.1, it is not required that developers search for vulnerabilities in their TOEs. This then fails at promoting the developer to follow the good development practices (where vulnerability analysis should be included).

CC does provide a few limited provisions attempting to ad-dress implementation vulnerabilities. However, CC method-ology of vulnerability assessment is not rigorous to verify that a given product does not contain implementation vulnerabil-ities that could expose customers to risk. The guidance for vulnerability analysis, as described in the Common Evalua-tion Methodology,11 is too generic, abstract, and subjective. It does not provide any specific methods and procedures. Thus, the effectiveness level of the vulnerability analysis and pen-etration testing often varies, which inevitably makes the out-come of the vulnerability analysis subjective.

Due to lack of systematic analysis of the risks and threats to the TOE and its operational environment, the specification of security problems in ST is subjective and may leave some risks/threats out of consideration in the ST. The omission of certain risks or threats may lead to incomplete coverage of vulnerabilities.

Advice on how to improve Common CriteriaIt should be pointed out that the paramount goal of CC evaluation should be the promotion of good security prac-tices in the product development. Hence, developers should adopt good practices of security in the development life cycle in order to improve the security assurance of their products. Moreover, the advances of security in the development life cycle affords the developer to apply more good security prac-tices in the development process,12 while the fast evolving vulnerability attack landscape demands the developer follow the same practice as well.

11 “Common Methodology for Information Technology Security Evaluation, Evaluation methodology, Version 3.1 Revision 3, CCMB-2009-07-004.”

12 “Software Security Assurance – State of the Art Report”, Information Assurance Technology Analysis Center (IATAC) and Data and Analysis Center for Software (DACS).

Limitations of general evaluation methodologyCommon Criteria is generic and does not consider the specif-ic security requirements for different kinds of IT products. It is a commonly understood that there is not a one-size-fits-all solution to security assurance concerns. Hence CC is unable to offer a more prescriptive approach, like some other stan-dards such as FIPS 140-2. Therefore, CC evaluation method-ology is abstract and subjective.

CC evaluation tends to be more driven by security functional requirements, i.e., verification that the IT product meets its

claimed security functionality. However, it does not provide a sufficient confidence in the char-acteristics of software that are considered essential to its securi-ty.10 For example, the assurance activity of TOE design analysis can only achieve assurance that the SFRs are implemented as spec-ified. But it is not sure whether the implementation of SFRs may introduce flaws or vulnerabilities.

Specifically, CC does not rigorous-ly demonstrate that the TOE does not act unintendedly and that the TOE preserves the security prop-erties such as self-protection, non-bypassability, correctness, reli-ability, etc. Although the analysis of security architecture of TOE at-tempts to address such concerns, it is more or less an assertion of the developer that the product ex-hibits the aforementioned security

properties, which, however, cannot be rigorously verified.

Security target (ST) is the baseline of CC evaluation, which describes the security problems exposed to the TOE, the se-curity objectives to address the security problems, the secu-rity requirements to satisfy the security objectives, and the security functions to be implemented to meet the security requirements. However, the security problems and the secu-rity objectives are not identified based on a systematic way of threat and risk analysis. Instead, they are defined in the ST in an ad hoc way, which tends to be subjective. The downside of this is that it is hard for the evaluator to verify whether there are security problems left out of consideration, and whether there are missing security objectives which may leave the TOE exposed to potential threats.

CC allows developers to voluntarily restrict the scope of the product evaluation in the ST, and the definition of TOE’s evaluated configuration is at the discretion of the product vendor, which may or may not reflect the product’s default installation configuration or common configurations when

10 “Security in the Software Lifecycle”, Department of Homeland Security.

A vendor can even restrict the evaluation to certain security features and make certain assumptions about the operating environment and the threats faced by the product in that environment.

ISSA Journal | April 2011

©2011 Information Systems Security Association • www.issa.org • editor@issa.org • Permission for author use only.

27

Common Criteria | Changying Zhou and Stefano Ramacciotti

Considering the complaint about CC being costly and time consuming, we think it is partly due to the fact that the CC evaluation is not cost-effective – the benefit of improving the product security by evaluation is not commensurate with the time and money spent on the evaluation. Thus, our advice focuses on addressing part of the technical limitations iden-tified in the previous section and improving the effectiveness of CC evaluation by technical means, based on current deve-lopment of security in software development life cycle.

Specifically, it is well advised that additional good security practices be applied to the development life cycle, especially for those products undergoing high-level CC evaluation, such as EAL4. Tangible values would be added by the following:

• The developer performs threat and risk analysis at the beginning of the TOE development cycle, accompa-nied with misuse and abuse cases generated, and then submits the relevant evidence to the evaluator as a ba-sis for the vulnerability analysis

• The developer inspects the implementation to detect common security flaws, such as input validation, out-put validation, secure counterparts of insecure func-tion calls, system calls or libraries, etc.

• The developer performs vulnerability analysis, con-ducts risk-based testing and penetration testing, and then submits the reports to the evaluator as evidence.

Especially, the promotion of vulnerability analysis done by the developer shall force the developer to examine the prod-uct critically, from the attacker’s point of view. It is part of best practices the developer should adopt.

In addition, it will help the evaluator’s vulnerability analysis in two ways:

1. At evaluation level below EAL4, the low-level de-sign information and implementation details are not available. Meanwhile, the attack potential below EAL4 is assumed to be low. Hence, the evaluator can-not perform deep inspection and testing that requires sophisticated attack skills and resources. In this case the vulnerability analysis conducted by the developer may add additional assurance on the TOE.

2. At EAL4 or above, the low-level design information and implementation details are available. Moreover, the attack potential is supposed to be higher, i.e., the attacker may be equipped with sophisticated tools and deep technical expertise. Thus, the evaluator is expected to have an equivalent attack potential. Based on the developer’s vulnerability analysis, especially facilitated with specialized tools and sophisticated models for analyzing and testing the TOE which are developed by the developer, the evaluator is able to make a more meaningful and rigorous vulnerability analysis of his own. Thus, the evaluator’s vulnerabili-ty analysis can be improved significantly and be more effective.

To be transparent, the security measures which should be taken in product development, shall be described in develop-ment life cycle model, techniques, and tools used in the devel-opment process. It would be more efficient if verification of security assurance measures are part of the development life cycle process and conducted in site certification,13 as the re-sults of the certification may be applied to several CC evalu-ations for the same vendor and save the evaluation efforts.

Another way to improve CC evaluation is to augment or re-fine the CC assurance requirements to address the demands for adding good security practices into the development life cycle, as well as the special security requirements for different product types such as firewall, web application, database, etc. To avoid conflict of interest and maintain trustworthiness, meanwhile, subject individual product types to thorough risk analysis. Such additional assur-ance requirements shall be re-searched by experts, independent of specific product manufactur-er, and added to the protection profile (PP) corresponding to a product type.

As a matter of fact, a similar ap-proach has already been applied to the smart card evaluation. Among the official CC docu-ments, smart card evaluation is specified as a special group guid-ance documentation,14 which ba-sically includes an interpretation to account for the special secu-rity requirements for smart card evaluation.

ConclusionAfter many years of development, there are still many limitations in Common Criteria. It shall have to continuously improve to be relevant to current development of security assurance. Adoption of security practices into the development life cycle (e.g., threat and risk analysis, misuse and abuse case generation, analysis of implementation rep-resentation to detect any implementation defects, risk-based security testing, vulnerability analysis, and penetration test-ing, etc.) can not only improve the security assurance but also facilitate the evaluation process. All in all, the goal of improv-ing the security assurance cannot be achieved only through the third-party evaluation and certification;t it needs the

13 "Site Certification, October 2007, Version 1.0, Revision 1, CCDB-2007-11-001."

14 “Smartcard Evaluation, CCDB-2006-04-001”; “Application of Attack Potential to Smartcards, CCDB-2009-03-001”; “The Application of CC to Integrated Circuits, CCDB-2009-03-002”; “Requirements to perform Integrated Circuit Evaluations, CCDB-2009-03-003.”

The assurance activity of TOE design analysis can only achieve assurance that the SFRs are implemented as specified. But it is not sure whether the implementation of SFRs may introduce flaws or vulnerabilities.

ISSA Journal | April 2011

©2011 Information Systems Security Association • www.issa.org • editor@issa.org • Permission for author use only.

28

Common Criteria | Changying Zhou and Stefano Ramacciotti

Stefano Ramacciotti is the Director of the Italian Defense Common Criteria Testing Laboratory (Centro di Valutazione della Difesa, www.difesa.it/SMD/Staff/Reparti/II-reparto/CeVa) in Pisa, Italy. He holds a BS in Defense and Security from Pisa University (and Italian Naval Academy), is a Certified Information Systems Secu-rity Professional (CISSP), and a Common Criteria EAL4 Evaluator qualified by Canadian CSE. His areas of expertise include risk analysis, security policy development, PKI. He has contributed to numerous Defense and external publications and he is keynote speaker at various security con-ferences. He may be reached at ris.ceva@smd.difesa.it.

developer to reasonably retrofit and introduce good security practices into its product development life cycle.

About the AuthorsChangying Zhou, CISSP, is a senior IT se-curity specialist working for CSC Canada. His responsibility is leading Common Cri-teria evaluation of IT security products and providing technical advice and consultation regarding CC evaluation and security as-surance. Mr. Zhou has more than 10 years working experience in certification and validation of IT secu-rity products and software development of IT security systems. He has a Master’s degree in Information Security and may be reached at khzhou@hotmail.com.

ISSA Journal | April 2011

©2011 Information Systems Security Association • www.issa.org • editor@issa.org • Permission for author use only.