Safety and Security Considerations for Software and

Digital Hardware Verification

John Colley, Michael Butler

Verification Futures Conference

Reading, 6 April 2017

Acknowledgements

2

This work was made possible by the ATI SECT-AIR project funded by

Innovate UK. The participants in the SECT-AIR project are

OUR VISION OF AN AUTOMATED FUTURE

Zero accidents. Mobility for all.

www.intel.com

3

Introduction

• Motivation

• Standards and Guidelines

• System Theoretic Process Analysis (STPA)

• A Case Study: applying STPA and Formal Analysis

• Safety vs Security Considerations

• Organisational Concerns

• Summary

4

Motivation

• Safety and Security are emergent properties of the SYSTEM

– Must be verified at the system level

• Traditional verification techniques do not scale well to the system level

• Abstraction and Formal Analysis are increasingly being adopted

– Clock-Domain Crossing Verification

– C-RTL Formal Equivalence Checking

5

Where to Begin?

Formal Analysis

Abstraction

Safety

Security System Level Verification

6

Standards and Guidelines

• DO-326A (2014) Airworthiness Security Process Specification – Tackles Integration of Security Engineering with

Safety Engineering – Links the Security Process, the Safety Assessment

Process (ARP-4761) and the Systems Engineering Process (ARP-4754A)

• ISO/AWI 21434 (under development) Road Vehicles – Automotive Security Engineering

• NIST 800-160 (2016) Systems Security Engineering Considerations for a Multidisciplinary Approach in the Engineering of Trustworthy Secure Systems – Complements ISO/IEC/IEEE 15288:2015, Systems and

software engineering – System life cycle processes 7

NIST 800 160

“security is defined as freedom from those conditions that can cause loss of assets with

unacceptable consequences”

8

“the system protection capability is a system control objective

and a system design problem ”

NIST 800 160 “Trustworthiness”

“Trustworthiness requirements can include, for example, attributes of safety, security,

reliability, dependability, performance, resilience, and survivability under a wide

range of potential adversity in the form of disruptions, hazards, and threats.”

ASSURANCE CASE

“Effective measures of trustworthiness are meaningful only to the extent that the

requirements are sufficiently complete and well-defined, and can be accurately assessed.”

9

• Provides guidance to software developers wishing to use formal methods in the certification of airborne systems

• Identifies the

– Modifications

– Additions

to DO-178C

– Objectives

– Activities

– Software Lifecycle Data

including the artifacts expressed in formal notation and the verification evidence (Section 6 DO-178C) that could be derived from them

RTCA DO-333 Formal Methods Supplement to DO-178C

10

DO-333 Table FM.A-3

11

NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014

• Hazard Analysis Technique

– Identify potential causes of accidents: scenarios that can lead to losses

– Eliminate or Control hazards in design or operations before damage occurs

• Causal factors

– Design errors including software flaws

– Component interaction accidents

– Human decision-making errors

– Social, organisational and management factors contributing to accidents

• Also applicable for Security Analysis

System Theoretic Process Analysis (STPA)

Engineering a Safer World, Leveson MIT 2011 12

The Two Phases of STPA

1. Identify Potentially Hazardous Control Actions and derive the Safety Constraints

2. Determine how Unsafe Control Actions could occur

Engineering a Safer World, Leveson, 2012 13

The Two Phases of STPA

1. Identify Potentially Vulnerable Control Actions and derive the Trustworthiness Constraints

2. Determine how Untrustworthy Control Actions could occur

Engineering a Safer World, Leveson, 2012

14

Compliance Traceability System Requirements to High-Level Requirements

focusing on the objectives of DO-333 Table FM.A-3

We use System Theoretic Process Analysis

(STPA) and

Formal Modelling to demonstrate compliance

15

CASE STUDY

16

“.. FGS system has two physical sides, or channels, one on the left side and one on the right side of the aircraft. These provide redundant implementations that communicate with each other over a cross-channel bus.” NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014

Flight Guidance System Function

17

R1. At least one side shall be the pilot flying side.

R2. At most one side shall be the pilot flying side.

R3. Pressing the Transfer Switch shall always change the pilot flying side.

R4. The system shall start with the Primary Side as the pilot flying side.

R5. The system shall not change the pilot flying side unless the Transfer Switch is pressed.

System-level Requirements (stated informally)

NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014

18

System Level Model

19

Synchronous Implementation

20

NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014

• FGS has two physical sides, or channels, one on the left side and one on the right side of the aircraft. These provide redundant implementations that communicate with each other over a cross-channel bus.

• Each bus in the synchronous Pilot Flying example transfers its input to its output with a one-step delay

• Only the pilot not flying side listens for the Transfer Switch • If a side believes itself to be the Not_Pilot_Flying side, it will

become the Pilot_Flying side when it sees the Transfer_Switch pressed (i.e. a rising edge).

• If a side is the Pilot_Flying side, it will become the Not_Pilot_Flying side when it sees the other side become the pilot flying side.

Synchronous Implementation – Informal Requirements

21

NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014

Control Action Not Providing Causes Vulnerability

Providing Causes Vulnerability

Wrong Timing or Order (too early, too late)

Stopped too Soon or Applied too Long

Assert pilot flying V1 Request for control not issued when the transfer switch has been pressed

V3 Request for control issued when the transfer switch has not been pressed

V5 Too Early: As V3 V6 Too Late: Delay of requested switch

V9 Stopped too Soon: Other side does not see request for control V10 Applied too Long: As V2

De-assert pilot flying

V2 Side does not acknowledge that it has relinquished control when requested to

V4 In control but signal issued to the contrary

V7 Too Early: As V4 V8 Too Late: Delay of requested switch

V11 Stopped too Soon: As V2 V12 Applied too Long: As V6

STPA Step1: Identify Vulnerabilities

22

Vulnerabilities can lead to Failure • Loss of required function • Mode Confusion: the crew do not know what state the system is in,

which could lead to loss of the aircraft

C1. pilotflying must not be asserted by a controller unless

a rising edge has been received on transferswitch

and the controller’s state is not_flying V3, V5

C2. pilotflying must not be de-asserted by a controller unless

a rising edge has been received on the bus

and the controller’s state is flying V4

C3. pilotflying must be asserted by controller in the same cycle

when a rising edge is received on transferswitch

and controller state is not_flying V1,5,6,9,10

C4. pilotflying must be de-asserted by controller in the same cycle

when a rising edge is received on the bus and

controller state is flying V2,7,8,11,12

C5. controller l state and controller r state never the same V3-12 R1, R2

Controller Constraints

R5

R3

23

Refine System Model to two Synchronous Processes

⊆ Safety Constraint C5

lstate ≠ rstate

24

STPA Step 2: Determine how Untrustworthy Control Actions could occur

• In Step 1, we considered the Vulnerabilities

• In Step 2, we analyse the Threats that expose/exploit these Vulnerabilities

– Unintentional (safety)

– Intentional (security)

25

STPA Step 2: Determine how Untrustworthy Control Actions could occur

NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014

Spurious request to take control

Spurious response to relinquish control

SAFETY: Consider each vulnerability in isolation SECURITY: A concerted attack could exploit both vulnerabilities in sequence

26

Safety-Related Security The Policing Function

NASA/CR–2014-218244 Formal Methods Case Studies for DO-333 Darren Cofer and Steven P. Miller Rockwell Collins, Inc., Cedar Rapids, Iowa April 2014

Spurious request to take control

Spurious response to relinquish control

High Integrity Formal Model

Developed and Verified With Independence

27

Organisational Concerns

Project Development

Verification Function

Safety Function Security Function

Only the Verification Function can determine that the Safety AND Security requirements have been met

28

Summary • NIST 800-160 and DO-178C / DO-333 drive the

Development and Verification Processes • STPA provides a System-Level method for

– Identifying the Vulnerabilities – Determining the Safety and Security Constraints – Determining how unsafe control actions could occur

• The Policing Function – Is developed Formally from the Requirements – Can be used to monitor control processes which are too

complex to be verified formally – Considers multiple vulnerabilities to mitigate against concerted

attacks – Ensures that the Safety and Security Constraints are observed – Mitigates against mode confusion

• An Independent Verification Function – Ensures that Safety and Security Requirements have been met

29