WORK SAMPLE:

Safety Related Control System Audit

The following pages contain the safety related extracts from an audit report which examined a complete plant control system.

Client and Supplier have had a troubled contractual relationship, and Client is experiencing significant operational issues with the delivered plant and control systems.

All identifications have been removed from the extracts owing to the confidential nature of the report.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 11 of 90

2.1.2 Review of the Functional Safety Engineering Process and Deliverables

The review of the Functional Safety Engineering Process and Deliverables consisted of

reviewing information gathered on site and the available project engineering documentation

obtained from Client including design documents and QA/QC documentation. This

documentation was generally received in softcopy format. It is noted that other documentation

may exist however it was not practical or possible to obtain any additional documentation

during the timeframe of the Audit.

The review was performed with reference to AS4024.1503:2014 in accordance with email

agreements between Client and Supplier.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 19 of 90

3 Executive Summary

The safety PLC application program appears to be well structured and the coding appears to

be of a high standard. However the safety integrity achieved by software can only be verified

from a review of the software development process, and no documented evidence of the

software development process has been identified. In addition, there has been no

documentation presented to show the scope and the results of software functional testing

carried out.

There are some inadequacies of the hardware design of safety functions within the safety

PLC that warrant further consideration. There has been no review of maintenance

procedures at this time, but it is possible that maintenance procedures could be adjusted to

overcome the design limitations.

The functional safety validation reports are extensive but still have some limitations and

omissions that should be addressed. The hardware design inadequacies are glossed over,

whereas they should be considered in more detail, and their impact on maintenance test

requirements should be considered.

One of the most significant concerns identified is an apparent lack of Client involvement in the

risk assessments that the safety system design is based on. Client are legally responsible for

ensuring that adequate risk assessments have been carried out on machines they own and

operate, and for ensuring that the risks have been adequately mitigated.

Another major concern is that the safety PLC has been used for only a subset of the identified

safety functions, generally involving simple switch inputs. Other safety functions derived from

the operational state of the plant are implemented in the control PLC and are not adequately

protected against abuse or designated as safety functions.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 30 of 90

4.5 Safety System Engineering Process & Documentation

4.5.1 Limitation of Scope

The available Safety Instrumented System design documentation was reviewed. As stated

this review is based on the documentation available during the audit. It is possible that

additional and updated documentation may exist however it was not able to be located during

the timeframe of the audit.

4.5.2 Safety Management Plan

No documented evidence was found for the existence of a safety management plan, but it is

considered likely that Supplier would have had internal project documentation in place for this

purpose.

The contract included references to AS 61508 (functional safety) and AS 4024 (safety of

machinery). It is understood that Supplier requested acceptance of safety implementation in

accordance with AS 4024.1503:2014, which uses performance level (PL) to specify the

required safety integrity of safety functions. In the context of the ...... plant this was a

reasonable decision.

AS 4024 does not place the same up-front emphasis on functional safety management via a

documented plan and safety lifecycle as required by AS 61508. However it does specify

requirements for documentation of safety functions and does require a planned lifecycle

approach for development of the application program for a programmable electronic system.

AS 4024 emphasises risk assessment and design to mitigate the risks, but does not address

operation and maintenance of safety equipment. It is recommended that Client should

consider drafting a high level safety management plan to co-ordinate operational and

maintenance activities so that the ongoing function and safety integrity of the plant safety

systems can be monitored and audited. Guidance can be taken from AS 61508.

4.5.3 Risk Assessment Reports

There is a set of Supplier risk assessment reports, with each report identified as a part of

XXXXXX1 that covers the primary area, crushing area, scalping area and out loading area of

the ...... quarry. A separate report without document number covers the finishing area. The

reports are all marked with revision identifiers, but only the primary area report is dated within

the document.

The location of the risk assessment workshops is not identified in the reports, but from the

brief list of attendees without Client representation it appears that these may have been

conducted overseas.

The risk assessment reports are based on machine safety standards and consider mainly

personnel safety risks. They appear to be comprehensive and well presented, reflecting a

planned and managed risk assessment process. However the lack of Client representation

remains a concern since the reports appear to have formed the basis for the design of the

safety functions. These reports determine the required performance level (PL) for individual

safety related control functions.

Another risk assessment report has been prepared by Supplier using a third party Australian

risk assessment professional as the facilitator. This workshop session was carried out in

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 31 of 90

mmmm yyyy, and included Client personnel. It considered safety, financial and environmental

risks.

The yyyy risk assessment was conducted using the principles set out in risk management

standards, but does not mention or reference functional safety or machine safety standards.

It reflected a process functional safety approach where it considered the effects of equipment

malfunctions, but diverged from normal functional safety methodology in that it mixed the

failure of safety functions in with the failure of non-safety control functions. It did not address

any of the machine safety considerations for harm to personnel as set out in AS 4024, and

which are mandatory under workplace health and safety legislation.

The format of the yyyy report does not lead to determination of safety integrity requirements,

but it does note that various safety functions are SIL rated.

Of particular concern is the statement made in various places within the yyyy report that there

are SIL 4 protective functions in place for collision detection on pivoting conveyors. SIL 4

functions are extremely difficult to implement, and it could be expected that there would be

very onerous maintenance requirements. No reference to SIL 4 functions was found in other

safety documentation. It is hoped that this is a misunderstanding and should have been

Category 4 rather than SIL 4.

There are also notes that emergency stop functions are SIL 3 rated. Supplier make no claim

for SIL 3 in the validation documentation.

4.5.4 Scope and Extent of Safety Functions

One of the requirements of AS 4024:1503:2014 that is not addressed in the design

documentation available for audit is a clear delineation of the scope and extent of safety

functions.

The safety PLC handles mainly emergency stops and conveyer lanyards, hence it is clear that

everything within the safety PLC is safety-related.

However there are a number of other “safety functions” that are classified as requiring

Performance Level “c” (PLc) in the risk assessments that have been implemented in the

control PLC. These functions do not appear to be segregated and identified as safety

functions in either the PLCs or the SCADA displays.

This is an area where Client are exposed to a significant risk should an incident occur due to

a failure of the safety function implemented in the control PLC.

The following table lists the safety functions identified from the audit documentation and their

implementation in either a safety PLC or control PLC:

ID Safety Function

Highest Required Performance Level

PLC

SRCF 1a Plant E-Stop c Safety

SRCF 1b Emergency lanyards c Safety

SRCF 1c Local E-stop c Safety

SRCF 1d Global Plant E-stop c Safety

SRCF 2 Audible alarm for prestart warning. c Control

SRCF 3 Automatic stoppage upon downstream c Control

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 32 of 90

equipment fault.

SRCF 4a & 4b Material & level sensor stopping upstream feed.

c Control

SRCF 5 Hopper level sensor for automatic stoppage of equipment.

c Control

SRCF 6 Audible and visual warning before movement Stockpile conveyors

b Control

SRCF 7a Safety bump stops on both ends of trolley - Stockpile conveyors

c Safety

SRCF 7b Local e-stops on both ends of trolley - Stockpile conveyors

c Safety

SRCF 7c emergency lanyards on both ends of trolley - Stockpile conveyors

c Safety

SRCF 8

Restricted access to the stackers walkways via castell key on local control panel and access gate to isolate the slewing movement - Stockpile conveyors

a MCC / Control

SRCF 9 Slewing motors equipped with brakes to be released by energy.

c Control

SRCF 10 Wind speed measurement. Automatic movement and locking into parking position.

a Control

SRCF 11 Train location sensors enable conveyor motive drive contactor.

b Control

4.5.5 Safety System Hardware Design

Design drawings for emergency stop and lanyard inputs show conventional dual channel

circuits. However they do not reflect best practice for the following reasons:

In all cases the two inputs have been taken into adjacent channels on the same input

module. In order to mitigate against common cause failures, best practice is to wire

the two inputs to separate input modules.

It is common to use antivalent inputs for machine safety functions, with one contact

normally open and the other normally closed. Where both contacts are normally

closed, as is the case at ......, best practice requires that test pulses be applied to

each input at different times to improve the diagnostic performance of the function.

Test pulses have not been used on ...... safety inputs.

In view of the above inadequacies, it is not considered valid to claim Category 3 compliance

for these circuits as there are fault modes that will not be detected at or before each demand.

These design limitations also mean that maintenance testing procedures need to be

enhanced in order to ensure that all fault modes can be identified.

Design drawings for safety outputs show a hard-wired quadruple redundancy output

arrangement. Two outputs drive the main contactor for each drive via dual redundant

interposing relays. The other two outputs drive the under voltage trip on the main air circuit

breaker (ACB) feeding the MCC, again via dual redundant interposing relays. The states of

the interposing relays and the contactors are monitored for fault detection purposes.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 33 of 90

It should be noted that this arrangement can be relied on if and only if the contactor and circuit

breaker monitoring contacts are mechanically linked to the main contacts, and hence will

show that the main contacts remain closed in the event that the main contacts become

welded. Typically this is the case for contactors only if the contactors are designed as “safety

contactors”. For general purpose industrial contactors, the monitoring contact will follow the

coil energisation state regardless of whether the main contacts become welded. No check

has been carried out on the types of contactors in use at .......

4.5.6 Safety PLC Coding Standards

No documentation detailing safety PLC Coding Standards was provided for review as part of

the audit. In the absence of a documented safety PLC Coding Standard document the safety

PLC code has been reviewed based on best practice.

The safety PLC code is implemented as rungs of ladder logic using built-in Rockwell function

blocks. This reflects the restricted set of programming facilities available for GuardLogix.

While it is possible to develop “Safety Add-On Instructions” there is no evidence that this has

been done for the safety PLC application program.

The code appears to be well structured, and is very well commented.

While there is no documented evidence provided to demonstrate that the code has been

developed in accordance with an appropriate documented safety lifecycle plan, the

presentation of the safety PLC code appears to reflect best practice.

4.5.7 Performance Level Evaluation Reports – Individual Plant Areas

Each report evaluates the performance level achieved for the individual safety functions in the

respective plant area based on the hardware design.

A “calculated” result is shown for PFH and PL for each safety function, apparently derived

from the Sistema software tool. A Sistema report is also presented for each plant area, but it

contains only the calculated results. No reference is made to the basis of the calculations,

including such factors as diagnostic coverage assumptions and common cause assumptions,

or to the source failure rate data. As such, without a documented basis for the calculations,

the calculated results in the report cannot be verified.

In this context, the inadequacies identified in the hardware design (refer previous section)

need to be considered in the common cause failure mode analysis.

Sistema contains a database of component failure rates that can be used for evaluation of

high demand safety functions. These failure rates are based on wear-out mechanisms for

electro-mechanical devices that operate frequently.

In typical machine safety cases where devices operate frequently and there are adequate

diagnostics incorporated to detect faults at or before each operation of a safety function, the

concept of proof testing has limited relevance.

Proof testing is more relevant for low demand functions with limited diagnostics, where the

safety function components may remain in the same state for long periods until called on to

operate. The primary intent of proof testing is to flush out any failures that are not observable

in normal operation or detectable by diagnostics, but that may prevent operation of the safety

function when a demand occurs. Proof testing typically requires more detailed testing of the

individual safety function components than would be expected in a simple functional test of

the overall safety function.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 34 of 90

It is important to note that component failure mechanisms are very different for the high

demand and low demand cases, and wear-out failure rates cannot be sensibly applied to low

demand functionality.

The calculated proof test intervals within the reports are based on applying wear-out failure

rates to a simplified formula for calculating a time averaged probability of failure for a low

demand function, and the resulting statements of the proof test intervals required to maintain

claims for SIL ratings are considered to have very limited validity.

A more thorough validation of the design would consider the mode of operation of each safety

function, and would determine if high demand or low demand calculations were appropriate in

each case. The Sistema tool is appropriate for high demand functions only. If all functions

are high demand and the design incorporates the required level of diagnostics, proof testing

may not be relevant, and periodic functional tests may be adequate.

However the inadequacies identified in the hardware design (refer previous section) with

respect to inadequate diagnostics on dual channel input circuits do need to be addressed in

maintenance test procedures.

It should also be noted that the claim that safety functions implemented in the control PLC

have achieved PLc is not supported by the available audit documentation.

AS4024.1503:2014 clause 6.2 limits the claim for a PLC system to PLa or PLb. Achieving

PLc and above requires the use of “well tried” components, and for a PLC this requires

evidence that it has been “made and verified using principles which demonstrate its suitability

and reliability for safety-related applications”. While it may be possible to provide documented

evidence of this for a high quality PLC product, it is most simply achieved by using a safety

PLC that has been certified in accordance with IEC 61508.

In the absence of any supporting documentation from Supplier for either the control PLC

components or the engineering design and programming of the safety functions in the control

PLCs, the claim of PLc for safety functions implemented in the control PLC is not considered

credible or valid.

4.5.8 Electrical Functional Safety Report XXXXXXXX2

The report purports to present the following information for each safety function:

1. Specifications review.

2. Equipment under control (EUC) Risk Assessment.

3. Determine the required Performance Level (PLr) – From Supplier Risk Assessment.

4. Identify the safety-related parts which carry out the safety function and evaluate the

required performance level (PL) considering –

• Category.

• MTTFd (mean time to dangerous failure)

• DC (diagnostic coverage)

• CCF (common cause failure)

5. Verification of PL.

6. Validation – to ensure all requirements are meet.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 35 of 90

7. Proof Test Intervals.

8. Software Validation.

It is noted with reference to item 4 above that the “evaluation” is addressed in the individual

plant area reports discussed in the preceding section, and that the functional safety report

contains only a compilation of the results from the hardware evaluation reports.

The following errors and omissions have been observed from a review of the report:

The report misrepresents the significance of the software safety lifecycle V-model,

which is intended to reflect a planning and control mechanism for software

development. The report examines no evidence relating to software development,

but purports to show compliance by analysis of the finished product. The safety

integrity achieved by software is a reflection of the development and test processes,

and is not derived simply from an analysis of the finished product.

The report analyses the software code against a subset of the guidelines set out in AS

4024.1503:2014. It does not consider all of the guidelines, and does not consider the

requirements set out in the Rockwell GuardLogix Safety Reference Manual. Refer to

Appendix F for further details.

4.5.9 Protection of Safety Functions

Review of functional specifications and operating manuals has not provided any evidence of

an approach to the design of safety functions from an operational perspective. In this context,

best design practice would consider the following:

Plant operators need to have a clear understanding of how safety functions operate,

what impact they have on the process, and how they will be reported and alarmed.

The reset mechanism should be clearly designed and documented. AS 4024

contains a number of specific requirements pertaining to the reset function and these

should be captured in the safety function design documentation.

Facilities to override safety functions may be required to allow continued operation

under abnormal conditions.

It is important for the override functions to be well designed and documented so that they can

be managed and used with due consideration of the increased operational risks that will be

present. Ad hoc allowance of overrides in an uncontrolled manner is not permissible for

safety functions because it potentially leads to situations whereby personnel who would

normally be protected by a safety function are carrying out their work totally unaware that the

safety function is inoperative. This represents an unacceptable potential risk of injury or death

to personnel.

The design of safety function overrides should ensure that they can be effected only by

authorised personnel, either by a physical key lock or by appropriate login restrictions. Where

practical, placing a safety function in override should enforce complementary equipment

lockouts or restrictions in logic to minimise the operational risk associated with the override.

Wherever possible, there should be an indication at the relevant workplace to alert personnel

that the safety function has been disabled.

It is observed that the safety functions implemented in the safety PLC do not have any means

of overriding the safety functions via the HMI, and overrides would have to be set and

removed via the safety PLC engineering workstation. In general the use of the engineering

workstation for this purpose is not desirable because it allows unrestricted access to the

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 36 of 90

safety functions once it is in use. Alternatively operating procedures should make it

mandatory that all E-Stops and lanyards are operational before the plant is allowed to run.

The other issue of concern is the capability to override safety functions that are implemented

in the control PLC, but are not differentiated in any way as safety-rated functions.

A further consideration for these safety functions is that they are not unconditional safety

functions that will always be available to protect personnel. The following statement from a

performance level evaluation report is typical of a number of safety functions:

The function to override (inhibit) safety functions implemented in the control PLC via the

SCADA is detailed in the Operational Manual. This document details that this function is only

available to the following user accounts:

Manager - Operation user account

Supplier - Maintenance user account

The safety functions can be inhibited and the status of the inhibit viewed on the equipment

SCADA popup via the parameter tabs or on area inhibit SCADA pages. The safety functions

can be inhibited on a per sensor basis if the sensor is selected as present for the equipment

The Operation Manual contains the following warning regarding the use of the inhibit sensor

functionality:

“Note: The inhibitions have to be considered the risks for the people and

the consequences for the material. When the sensor is inhibited may cause

a disturbance in the regulation. Only the responsible must be authorized to

change these parameters.”

During the audit operations and maintenance staff advised that operators run the system

under a high level user account that allows the override of safety functions instead of the

Operator user account as intended by the design and detailed in the Operation Manual.

It was reported that on plant handover to Client operators could not get the plant to run with

Operator level access as multiple instruments had to be disabled and this required higher

level access. Due to this operators were given higher level access so that they can disable

devices but there is no approval or tracking process around this to identify why a sensor is

disabled. This has resulted in uncontrolled disabling of safety sensor.

The equipment SCADA popups also allow the sensors associated to be selected or

deselected as present for the equipment based on whether they are physically present that

that device. For example one conveyor may have one belt rip sensors whereas another

conveyor may have two built rip sensors. This selection/deselection is available on the

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 37 of 90

Parameters P&ID tab of the equipment SCADA popup and is required as the SCADA

interface for all equipment of a specific type is generic, refer to section 4.4.7 SCADA

Standards for more information. This functionality is not detailed in the Operation Manual and

the SCADA user access level required to select or deselect a sensor is also not detailed.

Note that when a sensor is deselected on the Parameter P&ID tab its text appears grey (not

black) on the equipment SCADA Popup Parameters tabs and on the area inhibits SCADA

page.

Client maintenance staff advised that they are unsure how to determine what devices require

what sensors on the Parameter P&ID tab of the Device SCADA Popup and therefore they are

unsure if the correct sensors are selected. Refer to section 4.4.2 P&ID Documentation for

audit review of the plant P&IDs.

As shown in the screenshots above it is possible that a sensor is deselected on the

Parameters P&ID tab of the equipment SCADA popup but not inhibited on the Parameters

tab. In this case Client maintenance personnel have advised that it is believed that the sensor

operation is disabled but the SCADA representation to operation and maintenance personnel

is confusing.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 69 of 90

9 Summary/Recommendations

The major findings of the audit are summarised in the following table and recommendations provided

for each items.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 75 of 90

Item #

Engineering Process / Deliverables

Documentation Available For Audit

Requirement / Reference

Observation Conclusion Operation/Maintenance Impact

Recommendation Priority

Safety Instrumented System (SIS)

21 Safety Management Plan, including Safety Software Lifecycle

No

Documented plan for

achieving required safety functionality and safety integrity / Best practice - also recommended in AS 4024.1503:2014

There is no evidence of a safety management plan in accordance with functional safety standards or a project execution plan in accordance with AS ISO 9001 within the documentation made available for audit.

Compliance with standards cannot be verified Safety system development may not have been managed in a controlled manner

Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice

Client should request sight of functional safety management or alternative quality management documentation from Supplier for further review. Client should also ensure that safety system operating and maintenance procedures are developed and audited based on an appropriate safety management plan for the site.

High

22 Risk Assessment including Client involvement

Yes

Plant owner / operator to conduct a risk assessment for every machine / WHS legislation

1. Risk assessment (yyyy) that Client participated in is not appropriate for this purpose. 2. Risk assessments prepared by Supplier are appropriate, but Client personnel do not appear to have been involved, either as participants or as reviewers.

Client is exposed to legal risk because of lack of attention to risk assessments

Noncompliance with WHS Legislation

Client should carry out a detailed review of the Supplier risk assessments based on their knowledge of plant operation and maintenance.

Critical

23 Safety System Design Yes

Clear differentiation between safety functions and between safety and non-safety functions / Selection of appropriate equipment for implementation of safety functions / AS 4024.1503:2014

E-Stops and lanyards are implemented in the safety PLC. Oher safety functions are implemented in the control PLCs, but are not clearly differentiated as safety functions in the PLCs or on SCADA displays.

The implementation of safety functions in the control PLCs is not satisfactory. The functions are not clearly identified as safety functions in the control PLC or on SCADA displays. The safety functions implemented in the control PLCs are disabled in some operating modes and are easily disabled in SCADA. The safety functions implemented in the control PLCs are unlikely to achieve the claimed safety integrity (performance levels) in operational practice.

Personnel are exposed to significant risks because the safety function operation has very low integrity.

Client should evaluate the specific risks that the control PLC safety functions are required to mitigate in further detail during the risk assessment review. If the required performance level of any safety functions remains at PLc or above as a result of the review, it will be necessary to consider implementing those safety functions in a safety PLC. Where safety functions may remain in the control PLC, modifications to PLC code and SCADA displays will be required in order to provide clear differentiation of safety functions from control functions. The review should also consider whether the SCADA override functions and the PLC mode disable functions are acceptable in terms of the consequent unmitigated risk levels.

Critical

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 76 of 90

Item #

Engineering Process / Deliverables

Documentation Available For Audit

Requirement / Reference

Observation Conclusion Operation/Maintenance Impact

Recommendation Priority

24 Safety System Hardware Design

Yes

Manage common cause faults in design / AS 4024.1503:2014

Dual channel inputs operating in valent mode (both normally closed) are wired to adjacent channels on the same input module.

Compliance with standards cannot be verified.

Potential for reduced integrity of safety function – claimed Performance Levels and Category may not be achieved in practice

In view of the significant effort that would be required to correct this, it is recommended that it is not changed, but that maintenance procedures be reviewed to determine whether common cause faults will be detected by routine testing, and whether any single common cause fault can disable the safety function.

Medium

25 Safety System Hardware Design

Yes

Provide adequate diagnostic functions in design – inputs / AS 4024.1503:2014

Dual channel inputs operating in valent mode (both normally closed) are wired to 24V supplies rather that to supplies with test pulses applied.

Compliance with standards cannot be verified

Potential for reduced integrity of safety function – claimed Performance Levels and Category may not be achieved in practice

It is recommended that test pulses be applied to dual contact inputs operating in valent mode.

Medium

26 Safety System Hardware Design

Yes

Provide adequate diagnostic functions in design - output monitoring / AS 4024.1503:2014

It is not confirmed whether output contactor / circuit breaker monitoring is via mechanically linked contacts or via a monitoring capability with equivalent effectiveness.

Compliance with standards cannot be verified.

Potential for reduced integrity of safety function – claimed Performance Levels and Category may not be achieved in practice

Review detailed circuit design and component specification to see whether CAT 3 requirements are achieved.

Medium

27 Safety System Hardware

Design Yes

Design to be validated by calculation of achieved performance levels. / AS 4024.1503:2014

Sistema safety software tool used for Performance Level calculations, but only the results are available for audit. There is no visibility of the input data and the assumptions made in the calculations. There are significant outstanding queries about the effectiveness of common cause design safeguards and diagnostic coverage that need to be factored into the calculations.

It is impossible to verify that the calculated Performance Level results are valid. Compliance with standards cannot be verified.

Hardware safety integrity performance of the safety

system hasn’t been adequately validated.

Client should request complete Sistema calculation files from Supplier to enable the validity of the calculations to be evaluated.

High

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 77 of 90

Item #

Engineering Process / Deliverables

Documentation Available For Audit

Requirement / Reference

Observation Conclusion Operation/Maintenance Impact

Recommendation Priority

28 Safety System Hardware Design

Yes

Maintenance requirements to be clarified / AS 4024.1503:2014

There appears to be confusion between routine testing and proof testing in the functional safety report. Machine safety circuits typically feature full diagnostics that will identify all fault events, but routine functional testing may be required for equipment that does not get operated frequently e.g. E-Stops. If diagnostics are inadequate, "proof testing" may be required to reveal undetected dangerous faults before they can cause loss of the safety function. Of necessity, "proof testing" is more complicated and extensive than routine functional testing as it needs to exercise each channel of redundant circuits separately and needs to exercise all credible fault scenarios. Because of the emphasis on diagnostics in machine safety standards, proof testing is not normally considered necessary for machine safety applications.

Compliance with standards cannot be verified

Maintenance procedures may not be adequate to ensure that the safety integrity of the system will be maintained at the specified Performance Levels. Need to distinguish between “routine” tests which are simple functional tests relying on diagnostics to ensure that all faults are detected, and

“proof” tests which exercise all potential operating modes of a safety function to ensure that all hidden faults have been revealed. “Proof” tests should only be required where diagnostics are inadequate to reveal all possible faults including common cause faults.

Client should request a detailed explanation from Supplier on the basis of their test requirement calculations. This explanation should also detail the failure rate data sources used for these calculations - wear-our B10 rates versus failure after prolonged lack of use.

High

29 Safety System Functional Design

No

Specification of safety functions / AS 4024.1503:2014

No documentation available for audit

Compliance with standards cannot be verified

System documentation inadequate for safe operation and maintenance of the plant

Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard.

High

30 Safety System Functional Design

No Manual reset function / AS 4024.1503:2014

No documentation available for audit

Compliance with standards cannot be verified

Reset function design / operation / integrity may not comply with requirements of standard

Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard.

High

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 78 of 90

Item #

Engineering Process / Deliverables

Documentation Available For Audit

Requirement / Reference

Observation Conclusion Operation/Maintenance Impact

Recommendation Priority

31 Safety System Functional Design

No

Muting or override functions / AS 4024.1503:2014

No documentation available for audit on the intended design of the override facilities. Override of safety functions implemented in the control PLCs is occurring in an uncontrolled manner and operators have SCADA log on access to disable safety functions.

Compliance with standards cannot be verified

Inadequate facilities provided to identify safety functions on SCADA displays and to highlight the potential consequences of disabled safety functions. Safety systems disabled in an uncontrolled manner and personnel exposed to risks that are not being managed by the disabled safety functions.

Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard. Client to:

Review and document all safety sensors and update SCADA equipment sensor selections

Review and document all disabled safety sensors

Put in place a system to control disabling of safety sensors

Restrict operator SCADA access so that only approved personnel can disable safety devices and that disabled sensors are regularly reviewed and faulty sensors replaced in a timely manner

Critical

32 Safety System Functional Design

No Fault considerations / AS 4024.1503:2014

No documentation available for audit

Compliance with standards cannot be verified

Safety system may not respond as required when fault conditions occur.

Client should request issue of functional specification documentation from Supplier to enable further review of safety function design against the requirements of the standard.

High

33 Safety System Application Program Development

No

Planning and documentation of software safety lifecycle / AS 4024.1503:2014

No documentation available for audit

Compliance with standards cannot be verified

Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice

Client should request sight of functional safety management or alternative quality management documentation from Supplier for further review. If the management plan cannot be provided for application program development, Client should consider a full review of the safety system software by a third party.

High

34 Safety System Application Program Development

No

Documentation of specification and design / AS 4024.1503:2014

No documentation available for audit

Compliance with standards cannot be verified

Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice

Client should request issue of software design documentation from Supplier to enable further review of software design against the requirements of the standard.

High

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 79 of 90

Item #

Engineering Process / Deliverables

Documentation Available For Audit

Requirement / Reference

Observation Conclusion Operation/Maintenance Impact

Recommendation Priority

35 Safety System Application Program Development

Yes Functional testing / AS 4024.1503:2014

Inadequate evidence of the required full scope of functional testing

Compliance with standards cannot be verified

Safety Systems may not function as intended

Client should request Supplier to provide evidence of test planning and recording that complies with all requirements of the standard. Client should undertake full re-commissioning of the safety functions to verify operation.

Critical

36 Safety System Application Program Development

No

Configuration management / AS 4024.1503:2014

No records available for audit

Compliance with standards cannot be verified

System documentation inadequate for safe operation and maintenance of the plant

Client should request Supplier to provide evidence of configuration management planning and recording for further review.

Medium

37 Safety System Application Program Development

No

Compliance with manufacturer's safety manual / Best practice AS 61508

No evidence available for audit

Compliance with standards cannot be verified

Potential for reduced integrity of safety functions – claimed Performance Levels and Category may not be achieved in practice

Client should request Supplier to provide evidence of compliance with the GuardLogix Safety Reference Manual for further review.

High

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 85 of 90

15 Appendix F – Functional Safety Analysis

15.1 Items Addressed in Electrical Functional Safety Report XXXXXXXX2

15.1.1 Example Programming Rules

The “example rules” presented in AS 4024.1503:2014 Annex J.4 are intended to provide

guidance on the issues that should be addressed in the functional safety management

planning and the software safety requirements specification that the application program is

derived from. Supplier have chosen instead to use them as a checklist for reviewing the

application program. All of the relevant items in Annex J.4 have been considered in the

Supplier report:

15.1.2 Application Program Development

Application program development and test requirements are set out in AS 4024.1503:2014

clause 4.6.3. Within this clause there are 10 sets of requirements designated as a) to j).

Supplier have chosen to use the points of section e) only as a checklist for software

verification.

Within the Supplier Report, the following “Analysis Result” entries are not considered valid:

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 86 of 90

The requirement for “justified or accepted coding guidelines” involves a much greater scope

than the use of standard function blocks. This refers to the requirements for project functional

safety documentation.

The requirement for “data integrity and plausibility checks” relates to defensive programming

requirements within the code, and again the implementation should be documented. It has

nothing to do with the validation and testing process, except that the robustness of the data

validity checking should be examined by testing with invalid data.

The requirement that “code should be tested by simulation” is specifically not related to field

testing.

15.2 Items NOT Addressed in Electrical Functional Safety Report XXXXXXXX2

15.2.1 Application Program Development

Application program development and test requirements as set out in AS 4024.1503:2014

clause 4.6.3 that are not addressed in the Functional Safety Report include:

Documentation – 4.6.3 a) and 4.6.3 g)

Software design – 4.6.3 b) and 4.6.3 c)

Verification and testing – 4.6.3 f) and 4.6.3 h)

Configuration Management – 4.6.3 i)

15.2.2 GuardLogix Safety Reference Manual

It is a fundamental requirement of functional safety standards that the usage of certified

equipment complies fully with the safety manual prepared by the manufacturer. For complex

equipment such as safety PLCs the manufacturers typically supply checklists to examine

compliance with the requirements of the safety manual. Best practice dictates that the

checklists should be filled out comprehensively and accurately, and it is reasonable to expect

that they should form part of the functional safety report.

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 87 of 90

Rockwell provide the following checklists for the GuardLogix safety PLC:

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 88 of 90

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 89 of 90

Control System Audit Report

For Client

xxxxxxxxx Rev 0 Page 90 of 90