Edgilis principles of isa may11

Principles of

Independent Safety

Assessment (ISA)

MAY 2011

Written by

Billy Fong

Senior Consultant

Edgilis Pte. Ltd.

E [email protected]

W edgilis.com

Abstract

Independent Safety Assessment (ISA) has become increasingly common for railway projects in recent years.

The main objective of commissioning an ISA is to provide assurance that safety management processes have

been adequately implemented and that the risk associated with the project has been reduced to a level As

Low As Reasonably Practicable (ALARP).

This paper details the ISA concept, the role of an ISA, the benefits of commissioning an ISA, and some

examples of ISA methodologies that can be adopted.

Principles of Independent Safety Assessment

edgilis.com | May 2011 of 8

Table of Contents

1. Introduction 3

2. What is ISA and Do I Need One? 4

3. Role of ISA 5

4. ISA Competency: who should do it? 5

5. ISA Methodologies 6

6. Closing Thoughts 7

7. References 8



1. Introduction

Rail is one of the most complex sectors, from both a technical and business standpoint. Railway

authorities, which are generally public institutions, have to leverage a supplier base composed of

worldwide technology suppliers in order to build rail networks which can compete with other

transportation modes. These systems, which are already increasingly complex, also need to be able to

operate with neighbouring railway networks, and to support a never-ending pursuit of higher

operational efficiency and decreased maintenance costs. Amidst all this, railway authorities are held

accountable for the safety of their networks and systems, with ever-more stringent safety regulations.

Assessing the safety of system is therefore a constantly renewed exercise during which railway

authorities must identify the main risks and ensure they are mitigated over time. There are a

seemingly endless number of books and manuals on system safety standards and best practices, but

experience demonstrates that even a strict enforcement of those stringent rules does not guarantee a

safe system. Indeed, applying rigorous requirement specification is a prerequisite, but does not

mitigate system risks to acceptable levels.

Some typical problem areas within system safety management are:

Inaccurate or incomplete identification of hazards and requirements;

Inappropriate depth of analysis;

Incomplete safety argument;

Inadequate evidence supporting the argument; and

Insufficient competency or experience of safety engineers.

One way of reducing the occurrence of such problems is to have an independent assessment of these

elements of the system safety process. Independent Safety Assessment (ISA) can play a major role in

ensuring that these elements are not compromised on a project. In order to provide this level of

assurance, the Assessor needs to access all the project material, as it is being produced, and

understand the design decisions that were made during the early stages of development.



2. What is ISA and Do I Need One?

ISA helps to make a judgement about the safety adequacy of a product, system or process in a

particular context and environment and against a set of requirements. During an ISA, it may be

required to demonstrate compliance to recognised standards. Depending on projects’ requirements,

full compliance may not be necessary.

The purpose of an ISA is to audit and assess processes used in a project to show compliance to best

and appropriate practice and to assess the adequacy of the evidence that has been generated during

application of those processes. An ISA offers an independent view of the safety processes on a project

based on experience and a thorough understanding of the relevant standards. The main objective is to

provide assurance that a contractor/supplier not only considers but also addresses safety issues

appropriately.

Other motivations or benefits for undertaking an ISA include:

To comply with a Standard that requires an ISA – for instance, when carrying out work in

accordance with Defence Standard 00-56 or for safety critical systems for the UK railway

industry;

To provide added confidence that safety claims are justified and that any weaknesses are

identified and dealt with, as it is done independently from existing safety analysis and

assessment; and

To demonstrate to a regulator that your system is safe – although there is no mandate for the use

of an ISA for all railway projects, there are advantages and benefits that can be gained from

effective use of the ISA roles (e.g. identifying and closing potential gaps at early design stages,

which is naturally more cost-effective than making a change in manufacturing or later project

stages; and providing assurance to stakeholders that a sound safety management is being

professionally managed for the project throughout the lifecycle).

In many cases, conducting an ISA is just good practice. Using an ISA may help the contractor/supplier

in safety planning and analyses. This tends to happen naturally during the audit/assessment process

during which the contractor/supplier is providing information to address the queries by the Assessor.

Additionally, during the early stages of a project, an Assessor can often provide generic guidance or

advice, as long as independence is not being compromised.



3. Role of ISA

Commissioning an ISA and defining its role should be done as early as possible in the project lifecycle

development.

Generally, the frequency and depth of the safety audits and assessments as well as the level of

independence of the Assessor is based on the varying levels of complexity and risk presented by the

project. Typically, projects of less complexity or with lower risks can be handled by a single assessor,

who may well be working for the contractor directly. However, undertaking safety audits and

assessments of very complex and high risk projects will likely to involve a team of assessors from an

independent organisation.

The organisation commissioning an ISA should prepare a remit with the requirements of the ISA

including, but not limited to:

qualifications, experience and level of independence of the Assessor including any references to

previous audits and assessments;

the scope of the audit/assessment. This could be limited to certain subsystems within a

system (e.g. subsystems that have undergone a design change since last release);

the purpose of the audit/assessment (e.g. as a supporting document to be submitted for

management for approval); and

the basis of the audit/assessment (e.g. the documents that the project will be audited against

and the safety management framework within which the project is being run).

The Assessor needs to be convinced that the process captures, understands and mitigates the hazards

and identifies safety requirements associated with a system. This is carried out by a review of the

safety analysis and supporting documents that leads to the development of the system Safety Case.

4. ISA Competency: who should do it?

An Assessor should be able to evaluate the safety activities free from conflicts of interest. Even if a

client is paying for the ISA’s services, there should be a level of professional independence such that

the ISA is not influenced by project timescale and pressure from management. Certain organizations

have built a track record for delivering sound and professional audits.



The Assessor has to provide an authoritative, expert opinion on safety, and therefore has to be properly

qualified. The Assessor needs both technical and managerial skills in order to plan, arbitrate,

moderate meetings, and defend his position in a firm but non-confrontational manner. A balanced

team should be managed and coordinated by a team leader and engineers qualified to provide in-depth

knowledge of the individual systems and functions.

Competency requirements for ISA generally include the following:

Technical competency in safety engineering, including knowledge of the principles and concepts

of safety management (e.g. ALARP, risk and safety requirements), and of the safety analysis

techniques (e.g. HAZOPs, QRA, and Hazard Log Management). The ability to judge the scope

and depth of analyses carried out is also important.

Technical competency in the application domain, which should cover an understanding of the

specific technologies used and their context in the particular domain. Assessors need to have

the engineering knowledge and relevant experience in the application area and technology.

Auditing and assessment competency in managing the various ISA steps from determining the

scope and objectives to collecting and analysing the evidence to support the expert opinion to

making a judgement on the safety of a system and being able to document the findings in a

clear and unambiguous manner.

Behavioural competence – Assessors will need to rely on their interpersonal skills, their ability

to communicate and interview personnel at all levels of the organisation and their reporting and

presentation skills. They must also have demonstrated their integrity and trustworthiness.

ISA organisations should be able to supply evidence of competence covering these attributes,

supported by verifiable examples, as part of their proposal when bidding for an ISA role.

5. ISA Methodologies

ISA comprises of two main activities:

Process review and auditing for compliance to standards and safety plan.

Independent analysis in order to assess the implementation and results of project safety tasks.

In many cases, ISAs are performed by a team, as opposed to a single person, in order to review the

technical data and the processes separately and to allow for a more effective peer review. To



supplement the basic ISA methods of auditing and assessment (i.e. documentation review), the

following tools can also be applied:

Sampling – Assessing all the related evidence may not be practical on some projects, for

instance for a well-defined engineering process which generates large volumes of evidence or

documentation (from the Hazard Analysis and Hazard Log, FMECA, etc), which is an opportunity

to use sampling. Should the sampling reveal significant problems or issues, then further

detailed assessment could be conducted.

Vertical Slice Analysis – The objective of this activity is to trace the mitigation of a hazard

throughout the system lifecycle. The Assessor assesses the safety and design requirements

derived from a particular hazard, the specifications for implementation and the supporting

verification and validation evidence. The Assessor then builds an overall picture of how the

safety argument for a particular hazard was developed throughout the lifecycle. This approach is

useful when assessing a hazard of particular concern, or when assessing the overall

effectiveness of a project’s system engineering process.

Diverse Analysis – This can increase confidence in some critical aspect by performing an

analysis that differs from the one performed by the Project. The analysis is not entirely

repeated, but conducted enough to gain confidence under review. For example, the ISA Team

could perform an independent HAZOP on an area of particular concern and compare the results

with the Project’s analysis. This may increase confidence significantly more than an individual

assessor reviewing the Project’s HAZOP report.

6. Closing Thoughts

Early ISA involvement in a project can identify potential risks, especially with complex systems.

Effective usage of the ISA role can help to significantly de-risk a project. ISA increases the ability to

deliver a system in-line with international standards for Safety and RAM, they help to identify and

closes potential gaps in the Railway Authority’s requirements at the early design stages, when design

changes are easier and more cost-effective to implement and they help Railway Authorities gain

confidence that the as-built system will meet their aspirations.



7. References

i. Engineering Safety Management (The Yellow Book), Fundamentals and Guidance Issue 4.

ii. MOD, Def Stan 00-56/2, Safety Management Requirements for Defence Systems, 13th

December 1996.

iii. IEE/BCS, Safety, Competency and commitment, competency guidelines for Safety related

systems practitioners. 1999.

8. Author Biography

Billy Fong

Senior Consultant

Edgilis Pte. Ltd.

3 Fusionopolis Way

Symbiosis #05-20

Singapore 138633

T +65 6304 5311

F +65 6467 8900

E [email protected]

Billy is currently appointed as the team leader responsible for

managing the System Assurance Centre of Excellence within Edgilis.

He leads a team of System Assurance specialists in delivering a wide

range of RAMS services to various industries.

Billy has acquired significant experience in performing RAMS

studies/activities across a variety of railway projects in a number of

countries including Australia, Dubai, Johannesburg, Hong Kong,

Taiwan and Singapore.

In the delivery of these projects, he has undertaken an assortment of

project roles including Project Manager, Project Coordinator, Lead

Safety Consultant/Analyst, System Assurance Manager and

Independent Safety Assessor.

Edgilis principles of isa may11

Education

Transcript of Edgilis principles of isa may11