Dynamic Analysis of Safety Critical Software

8/4/2019 Dynamic Analysis of Safety Critical Software

1/4

Dynamic Analysis of Safety Critical Software

From the desk of

Sandeep Patalay

Dynamic Analysis of

Safety Critical Software

Sandeep PatalaySenior Systems Engineer, CMC Ltd

[email protected]


2/4


Background:The dynamic analysis of the safety critical software is an important phase

of the Independent Verification and Validation of the system. The EN 50128 has detailedthe methods that shall be used for this phase of the verification life cycle. This phase is so

critical to the project output that it demands meticulous planning and organization. Here

we discuss the dynamic analysis methods suggested by the CENELEC standards for SIL

4 software.

Boundary Value Analysis

The aim of this method is to remove software errors occurring at parameter limits or

boundaries. The input domain of the program is divided into a number of input classes.The tests should cover the boundaries and extremes of the classes. The tests check that

the boundaries in the input domain of the specification coincide with those in the

program. The use of the value zero, in a direct as well as in an indirect translation, is

often error-prone and demands special attention:

Zero divisor;

Blank ASCII characters; Empty stack or list element; Null matrix; Zero table entry.

Normally the boundaries for input have a direct correspondence to the boundaries for the

output range. Test cases should be written to force the output to its limited values.

Consider also, if it is possible to specify a test case which causes output to exceed thespecification boundary values. If output is a sequence of data, for example a printed table,

special attention should be paid to the first and the last elements and to lists containing

none, 1 and 2 elements.

Error Guessing

The aim of this method is to remove common programming errors. Testing experienceand intuition combined with knowledge and curiosity about the system under test may

add some uncategorised test cases to the designed test case set. Special values or

combinations of values may be error-prone. Some interesting test cases may be derivedfrom inspection checklists. It may also be considered whether the system is robust

enough. Can the buttons be pushed on the front-panel too fast or too often? What happens

if two buttons are pushed simultaneously?

Error Seeding

The aim of this method is to ascertain whether a set of test cases is adequate. Someknown error types are inserted in the program, and the program is executed with the test

cases under test conditions. If only some of the seeded errors are found, the test case set

is not adequate. The ratio of found seeded errors to the total number of seeded errors is anestimate of the ratio of found real errors to total number errors. This gives a possibility of

estimating the number of remaining errors and thereby the remaining test effort.


3/4


The detection of all the seeded errors may indicate either that the test case set is adequate,or that the seeded errors were too easy to find. The limitations of the method are that in

order, to obtain any usable results, the error types as well as the seeding positions must

reflect the statistical distribution of real errors.

Performance Modelling

The aim of the method is to ensure that the working capacity of the system is sufficient to

meet the specified requirements. The requirements specification includes throughput and

response requirements for specific functions, perhaps combined with constraints on theuse of total system resources. The proposed system design is compared against the stated

requirements by:

Defining a model of the system processes, and their interactions, Identifying the use of resources by each process, for example, processor time,

communications bandwidth, storage devices etc),

Identifying the distribution of demands placed upon the system under average andworst-case conditions,

Computing the mean and worst-case throughput and response times for theindividual system functions.

For simple systems, an analytic solution may be possible whilst for more complex

systems, some form of simulation is required to obtain accurate results. Before detailedmodelling, a simpler resource budget check can be used which sums the resources

requirements of all the processes. If the requirements exceed designed system capacity,

the design is infeasible. Even if the design passes this check, performance modelling mayshow that excessive delays and response times occur due to resource starvation. To avoid

this situation engineers often design systems to use some fraction (e.g. 50 %) of the total

resources so that the probability of resource starvation is reduced

Equivalence Classes and Input Partition Testing

The aim of this method is to test the software adequately using a minimum of test data.

The test data is obtained by selecting the partitions of the input domain required toexercise the software. This testing strategy is based on the equivalence relation of the

inputs, which determines a partition of the input domain.

Test cases are selected with the aim of covering all subsets of this partition. At least one

test case is taken from each equivalence class. There are two basic possibilities for inputpartitioning which are:


4/4


Equivalence classes may be defined on the specification. The interpretation of thespecification may be either input oriented, for example the values selected aretreated in the same way or output oriented, for example the set of values leading

to the same functional result; and

Equivalence classes may be defined on the internal structure of the program. Inthis case the equivalence class results are determined from static analysis of theprogram, for example the set of values leading to the same path being executed

Structure Based Testing

The aim of this method to apply tests which exercise certain subsets of the program

structure. Based on an analysis of the program a set of input data is chosen such that a

large fraction of selected program elements are exercised. The program elementsexercised can vary depending upon the level of rigour required.

Statements: This is the least rigorous test since it is possible to execute all codestatements without exercising both branches of a conditional statement.

Branches: Both sides of every branch should be checked. This may be impracticalfor some types of defensive code.

Compound Conditions: Every condition in a compound conditional branch (i.e.linked by AND/OR is exercised).

LCSAJ: A linear code sequence and jump is any linear sequence of codestatements including conditional jumps terminated by a jump. Many potential

sub-paths will be infeasible due to constraints on the input data imposed by the

execution of earlier code.

Data Flow: The execution paths are selected on the basis of data usage forexample a path where the same variable is both written and wrote.

Call Graph: A program is composed of subroutines which may be invoked fromother subroutines. The call graph is the tree of subroutine invocations in theprogram. Tests are designed to cover all invocations in the tree.

Entire Path: Execute all possible path through the code. Complete testing isnormally infeasible due to the very large number of potential paths.

Dynamic Analysis of Safety Critical Software

Documents

Transcript of Dynamic Analysis of Safety Critical Software