FAA Information Technology- Information Systems Security R&D Workshop 2003 13 June 2015© 2002-3...

1April 18, 2023 © 2002-3 USC-CSE

FAA Information Technology-Information Systems Security R&D Workshop

2003

Extending COCOMO II to Estimate the Cost of Developing Secure Software

Edward Colbert, Murali Gangadharan,Donald Reifer, & Barry Boehm

{ecolbert, murali, boehm}@cse.usc.edu, [email protected]

2April 18, 2023 © 2002-3 USC-CSE

Outline

Why Extend COCOMO II for Security?

The COCOMO II & Modeling Methodology

New Security Cost Driver & its Factors

COCOMO II Cost Driver Values

Analysis of Security Impact On Other Drivers

Next Steps & Summary

3April 18, 2023 © 2002-3 USC-CSE

Why Extend COCOMO II for Security

Military projects have considered security in developing software since the early 1980s

Until recently commercial projects often gave it little weight

Threat to business-critical systems & private information has grown

– Security can no longer be ignored

Few cost models (including COCOMO II) include security factors

– Based 1980s military perspective (Orange Book)

– Developing secure systems has changed dramatically

4April 18, 2023 © 2002-3 USC-CSE

Adding Security to Cost Models

Several approaches for addressing security

– Modernize existing models

• Primarily by updating definitions of cost drivers

– Add new cost drivers to cost models

– Develop separate security model

5April 18, 2023 © 2002-3 USC-CSE

Adding Security to Cost Models

USC has taken intermediate approach– Add factor that addresses security from 3 viewpoints

• Development • Operational • Physical (Development Constraints)

– Include factors as appropriate to all COCOMO II family cost models

– Address both commercial & military projects regardless of• Size• Domain • Level of maturity

Ed Colbert

2nd sub-bullet?

6April 18, 2023 © 2002-3 USC-CSE

Outline

Why Extend COCOMO for Security?






7April 18, 2023 © 2002-3 USC-CSE

Project scale factors: maturity,risk, flexibility, teamwork & precedentedness

Software product, process, project & personnel costdrivers

COCOMO II Refresher

COCOMOII

Model

Software size estimate

Software organization’sproject data

Effort & duration estimates

Cost, schedule distribution by phase, activity, increment

COCOMO II recalibrated to organization’s data

PMestimated

A(Size)(SF) EMii

Effort in Person Month

SF: Scale Factors (5) EM: Effort Multipliers(17)

8April 18, 2023 © 2002-3 USC-CSE

COCOMO II Family of Cost Models

COCOMO IICalibration

Models

Risk & Tradeoff Models

Allocation ModelsSizing

Models

Reuse Models

COCOTS

COQUALMO

CORADMO

COSYSMO

OTHERS

9April 18, 2023 © 2002-3 USC-CSE

COCOMO II Modeling Methodology

Analyze Existing Literature

Perform BehavioralAnalysis

Determine Form ofmodel &Identify relative

significance of parameters

Perform expertJudgment, Delphi

Assessment

Gather ProjectData

DetermineBayesian

A Posteriori update

Gather more data;Refine model

10April 18, 2023 © 2002-3 USC-CSE

Outline







11April 18, 2023 © 2002-3 USC-CSE

COCOMO II Security Driver (SECU)

Viewpoints– Design & Development for Security – Operational Security– Physical Security (Development Constraints)

Driver Ratings– Low – Nominal– High– Very High– Extremely High

12April 18, 2023 © 2002-3 USC-CSE

Security Factors

Design & Development for Security– Effect of design methodologies & processes for development &

validation when security is factor

Operational Security– Effect of security policies, processes, tools & facilities that:

• Permit identification of security events • Define subsequent actions to identify key elements • Report pertinent information to appropriate individual, group, or process

Development Constraints– Constraints placed on development when protecting software

facilities:• From outside perimeter to inside office space• Includes all of information system resources

13April 18, 2023 © 2002-3 USC-CSE

Design & Development for SecurityRating : Low & Nominal

Low– No security requirements– No protection other than provided by execution environment

NominalRequirements Informal security requirements formulated for system

Design Analysis of security functions using–Informal functional & interface specification–Descriptive high-level design–Informal demonstration of corresponding pairs

Testing Developer tests implementation of requirements –Black box testing

Life-cycle controls Simple Configuration Management with version numbers

14April 18, 2023 © 2002-3 USC-CSE

Design & Development for SecurityRating : High

Nominal +

Requirements Fully defined external interfaces Informal security policy modeling

Design High-level design enforces securityInformal low-level design description

Testing Independent testing of all functional requirementsInspection of COTS/OSS source code if availableDeveloper vulnerability analysis

Life-cycle controls Detailed delivery & installation procedures–with well-defined security defaults

Developer-defined life-cycle model–with well-defined development tools

15April 18, 2023 © 2002-3 USC-CSE

Design & Development for SecurityRating: Very High

High+Requirements Semi-formal functional specifications

Semi-formal security policy modeling

Design Semi-formal high-level designSemi-formal Correspondence demonstrationModular implementationDynamic analysis for COTS/OSS

Testing Evidence of coverage for all developer test resultsTesting of high-level design Independent vulnerability analysis Independent validation of analysis

Life-cycle controls Partial automation of CM – with authorization control, problem tracking, & detection of

modificationMeasurable life-cycle model

– with compliance to implementation standards

16April 18, 2023 © 2002-3 USC-CSE

Design & Development for SecurityRating: Extremely High

Very High +Requirements Formal functional specification

Formal security policy modeling

Design Formal high level explanationFormal Correspondence DemonstrationStructured implementation with minimization of complexitySecure container for COTS & OSS

Testing Analysis of coverage of testsAnalysis & testing for insecure statesOrdered functional testing with tests of low-level designCovert channel analysis

Life-cycle controls Compete automation of CM – with coverage for developer tools

Measurable life-cycle model– with supporting empirical data

17April 18, 2023 © 2002-3 USC-CSE

Operational SecurityRating: Low & Nominal

Low– No organization-wide security policies – Ad-hoc security practices– Optional firewall & virus protection

Nominal

Administrative controls

Basic Security policies – inc.

• Password & Virus Protection policy• Network access & system use policy

Guidelines for administrators & usersGuidelines for regular backups of critical data

Technical Controls

Reasonable practices for– Checksum Verification– Firewalls– Operating system logging

Simple password-based authentication schemes

18April 18, 2023 © 2002-3 USC-CSE

Operational SecurityRating: High

Nominal +

Administrative Controls

Security policies include– User administration policies– Access administration policies

Documentation & logging of all security incidentsDiscretionary Access Control Policies for all resourcesSecurity Screening of all staff on project

– Initial background screening

Technical Controls

Media access controls– Marking, logging & Integrity verification

Strong communication security implemented with reasonable practices for

– Virtual Public Networks– Wireless Access– Active systems & network monitoring

Multi-factor authentication using passwords & smart-tokens

19April 18, 2023 © 2002-3 USC-CSE

Operational SecurityRating: Very High

High +Administrative Controls

Security policies include– Incident response policy– Data classification policy

Incident response teams formed to handle security breachesMandatory Access Control Policies to all resourcesProject staffing process considers

– Separation of duties– Access with least privilege

Technical Controls

Defense-in-Depth strategy to protect system with reasonable practices for

– Layered system monitoring with Intrusion Detection Systems– Write protected audit trails

Digital certificates & signatures used for – Authentication– Message integrity – Non-repudiation

20April 18, 2023 © 2002-3 USC-CSE

Operational SecurityRating : Extremely High

Very High +

Administrative Controls

Comprehensive Security policies including– Business continuity plans– Disaster recovery plans

Strong role-based access control for all resources

Technical Controls

Defense-in-depth strategy for protection is implemented with reasonable practices for– Protection against insider & outsider attacks

– Honeypots for proactive defense

Independent vulnerability analysis & penetration testsAll communications are encryptedMulti-factor authentication with biometrics

21April 18, 2023 © 2002-3 USC-CSE

Development ConstraintsRating Description

Nominal– None

High– All source materials are locked up when not in active use

Very High– High +

• Audited security markings in code

Extremely High– Very High+

• Multi-compartment developer communication constraints

23April 18, 2023 © 2002-3 USC-CSE

Draft Security Effort Distribution

6%6%

24%

24%

24%

48%

76%

30%

60%

12%6%

6%

0%

20%

40%

60%

80%

100%

120%

140%

160%

180%

Inception Elaboration Construction Transition

Security-Range

Security-Min

Base

• With large Standard Deviation

25April 18, 2023 © 2002-3 USC-CSE

Existing COCOMO DriversAffected By Security

Proposed security cost driver (SECU) constrain existing COCOMO II cost drivers

Security functions add to project’s size Increases project risk

RELY Required software reliability

CPLX Product complexity

DOCU Documentation match to life-cycle needs

SITE Multi-site development

TOOL Use of software tools

26April 18, 2023 © 2002-3 USC-CSE

Security Effect on Risk

Projects risk is increased when

– Security driver rating is >= high &

– Rating of following drivers are <= low or nominal

• ACAP : Analyst Capability

• PCAP : Programmer Capability

• APEX : Application Experience

• PLEX : Platform Experience

• LTEX : Language Experience

28April 18, 2023 © 2002-3 USC-CSE

Next Steps

Reach consensus on cost drivers – FAA Workshop with AS400 on security, July 03– Delphi run & calibration of factors in works

Initiate efforts to statistically validate accuracy of model– Survey available 160+ COCOMO project data – Perform initial calibration

Create enhanced COCOMO data collection forms– Gather security related efforts

Compare actual project data to expert opinions– Calibrate the model by weighting

• Actual data

• Expert opinions using Bayesian statistical techniques

29April 18, 2023 © 2002-3 USC-CSE

Summary Proposed extensions to COCOMO for development of

Secure Systems– Based on Common Criteria & DITSCAP– 1 Driver: SECU– 3 Factors:

• Development for Security• Operational Security• Physical Security (Development Constraints)

– Affects on other COCOMO II Drivers• RELY, CPLX, DOCU, SITE, TOOL

– Affects on Size– Affects on project risk

Hopefully stimulated your interest & motivated you to participate by sharing project data

FAA Information Technology- Information Systems Security R&D Workshop 2003 13 June 2015© 2002-3...

Documents

Transcript of FAA Information Technology- Information Systems Security R&D Workshop 2003 13 June 2015© 2002-3...