COTS Based System Security Economics- A Stakeholder/Value Centric Approach

Related tool demo session: COTS Based System Security Test-bed (Tiramisu) Tuesday at Davidson Conference Center

Yue ChenPhD Candidate in Computer ScienceAdvisor: Dr. Barry Boehm

941 W. 37th Place, SAL Room 330University of Southern CaliforniaLos Angeles, CA, 90089, USAPhone: (213)740-6470Email: yuec@usc.edu

©All rights are reserved by the authors

2

Agenda

Background Goal of Research Nature of the Problem T-MAP Framework Tiramisu tool Demo Model Applications Initial Validation Results Conclusions and Future Work

3

Background

Trends– Increasing usage of COTS software in IT systems– Increasing concerns on COTS software vulnerabilities

Challenges– Evaluating CBS security in business context– Benefit of security investment is difficult to measure– “Twenty percent of vulnerabilities caused eighty

percent of the security risk”, but, what are they?

4

Goals of T-MAP

T-MAP: Threat Modeling based on Attack Path analysis– A Stakeholder Value Centric Approach

Help making decisions on how much security investment would be optimal– Max security strategy– Max cost-effectiveness strategy

Help system designers understand the security of COTS combinations in early project life-cycle

Help network administrators determine vulnerability priorities

5

Permitted Ports

Firewall Wrapper

Software Applications, COTS

e.g. Windows Server 2003

e.g. IIS 6.0e.g. SQL Server 2000

IT Infrastructure

e.g. Web Server e.g. CRM Server

Nature of The Problem

Org. ValuesProductivity Reputation

e.g. Regulatory

Vulnerabilities impactingconfidentiality, availability,integrity

Attacking PathsUnblocked vulnerabilities

Blocked vulnerabilities

6

T-MAP Framework

Three key steps:– Step 1: Interview with key stakeholders to determine

how organizational value rely upon IT security– Step 2: Enumerate what are the scenarios that COTS

system vulnerability can compromise organizational values

– Step 3: Evaluate the severity of each scenario by weights, and model COTS system security threat with total weights of all scenarios

Step 2 and 3 are tool automated (Tiramisu)

7

USC-ITS Server X Case Study – Background

Security protection of Server X, a sensitive database Determine best practice under limited budget Key stakeholders: students, faculties, staff Organizational goals

– Productivity of the teaching and research community– Regulation compliance– Privacy of students, faculties, and staff

COTS software installed on Server X:

8

Step 1 – Determine stakeholder/value dependencies on IT security

Evaluate the severity of security hazard scenarios by stakeholder/value impacts

Involves both qualitative and quantitative criteria Technical approach: Figure of merits and Analytical

Hierarchy Process (AHP) Example output (from USC Server X Case Study)

9

Determine the Weights - AHP Pair-wise Comparison

Example – Stakeholder value priority weights:

Reading: regulation is “very strongly” more important than

productivity

10

Step 2 – Attack Scenario Analysis

Enumerate the scenarios how an attacker can compromise stakeholder values through COTS system vulnerabilities

Attack Graph is established based on a comprehensive COTS vulnerability database involves 18,800 known vulnerabilities reside in 31,713 COTS software

11

Step 2 (Continued) – Example Output and Observations

Example out put of Step 2 (Tiramisu screenshot below)(Example output – from USC Server X Case Study)

12

Step 3 – Security Scenario Severity Evaluation

Severity Drivers Stakeholder value impacts Vulnerability technical

attributes– Impact on confidentiality,

integrity and/or availability– Remotely exploitable– Require valid user

account on victim host– Needs user activities

Attackers– Group size– Skill level– Motivation to attack

13

Step 3 (continued) T-MAP Severity Rating System

Severity Weight of Attack Path P:

Overall Security Threat Score of COTS System G:

ThreatKey of elements in Attack Graph:

Effectiveness of Security Practice:

14

Tiramisu Tool Demo

Tiramisu is the software implementation of T-MAP

15

T-MAP Applications (1) Security Investment Effectiveness Estimation

* Case study results estimated by professional security manager at USC-ITS

How much security threats can be avoided by implementing Firewall, Software hardening (patching), user account control, or file system encryption?

Results as well depends on the total value of the protected system

16

T-MAP Applications (2) Security Patching Economics

Prioritize COTS Based System vulnerabilities under business context– “20% percent of vulnerabilities causes 80% of the

security risks”, T-MAP tells what are the 20% Rational: Prioritize vulnerabilities with its ThreatKey; Example screenshot:

17

T-MAP Applications(3) COTS Security Economics

Economic curve of security patching(from USC Server X case study)

Sweet spot to invest in security Also driven by the total value of system

(from USC Server X case study)

Sweet spots to invest

18

Initial Validation Results

Vulnerability priority comparison:Security Manager’s manual results vs. Tiramisu results

Tow case studies conducted at USC Information Technology Services Division

Two more case studies in progress with:– Manual Art Senior High School– African Millennium Foundation

19

Limitations

Only sensitive to known COTS vulnerabilities– Empirical study by Arora shows that the average attacks per host per

day jumped from 0.31 to 5.45 after vulnerability get published

Only cover “one-step-attacks” that exploiting COTS vulnerabilities

Depends on comprehensive vulnerability database– Our database: 188,000 vulnerability published from 1999-2006 that

resides in 31,313 COTS software

Cannot effectively address passive attacks such as Phishing

20

Conclusions

A COTS security evaluation framework that captures stakeholder value propositions

Distill the potential impacts of thousands of vulnerabilities into management friendly numbers at a high-level

Results are organizational IT infrastructure specific

21

Future work

Explore applying game theory in T-MAP We are looking for real-life projects/system to further

validate and mature the framework Close integration with risk driven win-win spiral process

to engineer more secure COTS Based System (CBS)– Proactively evaluate CBS security in early life-cycle– Making convincing security business case for CBS– Help make better security protection plan

Contact: Yue Chen, yuec@usc.edu