audit Skills Training Handbook

HSE Auditing: Fundamentals, Skills and Techniques for Team Members

NPC Course Manual 2005

Arthur D. Little Limited Science Park, Milton Road Cambridge CB4 0XL Telephone 01223 392090 Fax 01223 420021 Reference 20365

Table of Contents


Chapter Page

No Overview of Environmental, Health, and Safety Auditing 5 Audit Approach 12 Basic Steps in the Typical Audit Process 23 Pre-Audit Activities 25 On-site Opening Activities 42 Understanding HSE Management Systems 55 Effective Interviewing 75 Preparing Working Papers 105 Assessing Strengths and Weaknesses 135 Gathering Audit Evidence 161 Sampling Strategies 189 Evaluating Audit Results 220 Writing Audit Findings 242 Post Audit Activities 260 Appendix A – Confirmation Letter Appendix B – Roles and Responsibilities of the Audit Team Appendix C – Guide to Acronyms

List of Exercises


Exercise Page

No 1 Understanding HSE Management Systems 69 2 Effective Interviewing - Difficult Interview Situations 88 3 Effective Interviewing - Conducting Interviews 93 4 Preparing Working Papers 126 5 Assessing Strengths and Weaknesses 152 6 Gathering Audit Evidence -

Developing Verification Strategies 177

7 Sampling Strategies 1 190 8 Sampling Strategies 2 200 9 Evaluating Audit Results -

Specific ‘Local’ vs ‘Report’ 230

10 Writing Audit Findings -

Critiquing Audit Findings 255

Overview of Health, Safety and Environmental Auditing




Recognising auditing as a powerful tool for managing and communicating HSE performance, organisations around the world are developing audit programmes to: • Provide assurance to corporate officers about the company’s

compliance status with HSE requirements and good industry practices.

• Assess potential HSE liabilities. • Demonstrate effective management of HSE obligations to

companies’ key stakeholders.

Annual Report Audit Report

Community

Investors/Shareholders

CEO and Management

Facilities

Audit Programme

An increasing number of companies are including status reports on their HSE audit programmes in their annual reports and/or annual environmental reports (e.g., Union Carbide, ARCO Chemical, NOVA, Deere & Company, WMX, DuPont, etc.).



Moreover, there are many sound business benefits of performing HSE audits.

Improve public image

Increase awareness and understanding of HSE hazards

Reduce employee and community exposure hazards

Improve compliance

Reduce costs by operating efficiently and safely

Reduce exposure to fines

To addressstakeholder

needsWhy Audit?

Overview of Health, Safety and Environmental Auditing – What is Health, Safety and Environmental Auditing?


Employee Health and Safety

Employee Health and SafetyCompliance ManagementCompliance Management

Hazard ManagementHazard ManagementEnvironmental ProtectionEnvironmental Protection

Management ToolManagement Tool

Overview of Health, Safety and Environmental Auditing – Definition of Health, Safety and Environmental Auditing


Auditing has become recognised throughout the world by various organisations as a useful tool in managing HSE issues. Although a variety of definitions have been established for HSE auditing, they all share common elements and themes. For example, auditing has been defined as…

• A systematic, documented, periodic, and objective review by a regulated entity of facility operations and practices related to meeting environmental requirements. (U.S. EPA Policy Statement on Environmental Auditing, July 1986)

• Internal evaluations by companies and governmental agencies to verify their compliance with legal requirements as well as their own internal policies and standards. (Environment Canada, May 1988, Environmental Protection Act, Enforcement and Compliance Policy)

• A series of activities undertaken on the initiative of an organisation’s management to evaluate environmental performance. (International Chamber of Commerce)

• An activity directed at verifying a site’s or organisation’s environmental, health, or safety status with respect to specific, predetermined criteria. (U.S. Environmental Auditing Roundtable)

Overview of Health, Safety and Environmental Auditing – Definition of Health, Safety and Environmental Auditing


…and the definition of auditing has broadened over the years to include the evaluation of management systems in determining a facility’s HSE performance status.

• A systematic, documented verification process of objectively obtaining and evaluating evidence to determine whether specified environmental activities, events, conditions, management systems, or information about these matters, conform to audit criteria, and communicating the results of this process to the client. (International ISO 14000 Standard)

• A management tool comprising a systematic, documented, periodic, and objective evaluation of the performance of the organisation, management system, and processes designed to protect the environment with the aim of: facilitating management control of practices which may have impact on the environment; and assessing compliance with company environmental policies. (Eco-Management and Audit Scheme—European Union)

• A systematic evaluation to determine whether or not the environmental management system and the environmental performance it achieves conform to planned arrangements, and whether or not the system is implemented effectively, and is suitable to fulfill the organisation’s environmental policy and objectives. (ISO)

Overview of Health, Safety and Environmental Auditing – Standards Against Which to Audit


Since auditing has emerged as a systematic process intended to verify compliance with established standards and, in some cases, to review the effectiveness of management systems, it tends to be most effective for those HSE issues that are well defined by specific audit criteria.

Audit Criteria

Best Management Practices

Facility Standard Operating

Procedures (SOPs)

Corporate/Division Policies and Procedures

ManagementSystems

Laws and Regulations

(Federal/National, State/Provincial,

and Local)

Increasingly, management system requirements are being incorporated into the scope of the audit in recognition that well-designed and well-implemented management systems are an important vehicle for maintaining compliance over time.

Audit Approach


Audit Approach


Most companies engage in a three-phased audit process to fulfill the objectives of the audit programme.

Post-Audit Activities

On-Site Activities

Pre-Audit Activities

Three-PhasedAudit Approach

Audit Approach



The pre-audit activities are designed to ensure that the audit team members and facility personnel understand the audit process, and their roles and responsibilities within that audit process, and are prepared to implement them.

Pre-Audit Activities Outcome

1. Initial planning activities

2. Document review

3. Audit plan preparation

Identification of key site issues

Preparation of detailed plans

Audit Approach


On-Site Activities

Arthur D. Little has developed a well-defined and systematic five-step process to provide organisation and structure to the on-site activities. This five-step process facilitates a review of how a facility manages its HSE obligations.

Basic Step

On-Site Activities Outcome

Opening meeting Tours Initial Interviews Document review

Strong working knowledge of key systems on site

Identification of key issues to review

Review of Step 1 information

Team meetings

Develop verification strategies

Reallocate team resources, if required

Identify potential impacts and management system weaknesses

Physical inspections Focused interviews Data and records

examination Verification testing

Analyse site programs Develop evidence to

substantiate findings Confirm status of

compliance Review data collected

Review factual accuracy of findings

Analyse/integrate findings of team

Prepare draft findings Confirm accuracy Identify potential root

causes

Daily debrief meetings Close-out meetings

Early, clear, consistent communication

Understand facility concerns Prepare preliminary draft

report

Step 1: UnderstandManagement Systems

Step 2: Assess Strengths & Weaknesses ofManagement Systems

Step 3: Gather AuditEvidence

Step 4: Evaluate AuditResults

Step 5: Report AuditFindings

In addition, the five-step process has been recognised by: • Hundreds of companies • Environmental Auditing Roundtable (EAR) • Canadian Environmental Auditing Association (CEAA) • International Chamber of Commerce in its Guide to Effective

Environmental Auditing • European Community in the Eco-Management and Audit

Scheme (EMAS)

Audit Approach


Two Components of HSE Auditing

There are two important components in completing the five-step process: • Assessment—Process to develop an opinion (judgment) on

the strengths and weaknesses of the activities under review. • Verification—Process to determine adherence to specific

standards. Although assessment- and verification-based techniques play a key role in the audit process, they each provide the auditor with different information.

Principal Activities What you look for Output

Deficiencies, problems, risks, conformance with good practice (performed during Steps 1, 2 and 4)

Professional opinion as to performance with regard to accepted practice and recommendations for improvement

Evidence of compliance with regulations, policies, and procedures (performed during Step 3)

Statement of performance against standards with identification of shortcomings/areas for improvement

Assessment

Verification

Auditors need to use a mix of assessment and verification to complete the five-step process. Some skills are better suited toward assisting an auditor in the assessment stage while others suit the verification stage. Assessment is dependent upon the auditor’s knowledge of: • Site operations • Management systems (e.g., policies, procedures, etc.) • HSE requirements • Environmental technology Verification is dependent upon the auditor’s knowledge of: • Auditing skills and techniques • HSE regulations • Internal standards

Audit Approach


The balance between assessment and verification will vary depending on where you are in the five-step process.

Five-Step Audit Process

Step 5: Report Audit Findings

Step 2: Assess Strengths andWeaknesses of Management Systems

Step 3: Gather Audit Evidence

Step 4: Evaluate Audit Results

Step 1: UnderstandManagement Systems

Assessment-related activities Verification-related activities

Based on the information gathered, the auditor may move forward in the process or may need to reassess the information gathered in previous steps. For example, results obtained during gathering audit evidence (Step 3) or evaluating audit results (Step 4) can lead an auditor back to reassess his/her understanding of the management systems in place and/or the strengths and weaknesses of those management systems. The resulting finding may be one that identifies gaps in the facility’s management systems or the proximate or root cause of the finding.

Audit Approach


Audit Skills, Techniques, and Tools

In implementing the five-step audit process, there are some essential audit skills, techniques, and tools that can be utilised to increase on-site efficiency and effectiveness.

Audit Skills and Techniques

Conducting interviews

Documenting the audit (working papers)

Writing audit findings

Gathering audit data

Using sampling strategies

Communicating audit resultsProtocols

Pre-audit

questionnaires

Discussion guides

Audit Tools

Protocols

Pre-audit

questionnaires

Discussion guides

Audit Tools

Audit Approach



The purpose of the post-audit activities is to ensure that: • Audit results are communicated to the facility and appropriate

levels of management. • Audit findings are addressed.

Post-Audit Activities Outcome

1. Develop report

2. Distribute report

3. Develop and implementcorrective actions

Audit report (performance status)

Corrective action plan

4. Track corrective actions

Status reports

Audit Approach - Context for Health, Safety and Environmental Auditing


Although HSE auditing has been recognised as a prominent and useful HSE management tool, it is not the only means by which an organisation manages its HSE issues. Auditing is one of many key activities within the typical HSE management processes.

Typical HSE Management Process

Identifying/managing issuesIdentifying/understanding requirementsIdentifying/evaluating risks

Influencing issues/requirementsSetting policy directionEstablishing performance standards/guidanceObtaining needed permits/approvals

Managing complianceManaging significant risksPreventing/reducing unwanted impactsRemediating past damagesResponding to emergencesImproving value to final customers

Measuring performanceAssuring performanceCommunicating performance

Auditing

Assessing Planning Implementing Reviewing

Supporting

Training & awareness Documenting / record keeping Managing information

Sound HSE management processes should include activities that address all four HSE management processes (assessing, planning, implementing, and reviewing) and incorporate the three key supporting activities (training and awareness, documenting/recordkeeping, and managing information). These same management processes are typically present at facilities in order to meet their HSE obligations.

Audit Approach - Context for Health, Safety and Environmental Auditing


Auditing can provide a basis for guiding, measuring, and evaluating the performance of a facility’s HSE management activities. In general, an audit looks at and evaluates the management processes to comply with HSE requirements for each functional area being reviewed (e.g., air pollution control, hazardous waste management, industrial hygiene, employee safety, etc.). The specific activities that will be audited will depend upon the programme objectives. For example:

Typical HSE Management Process

Identifying confined spacesIdentifying jobs/tasks

Developing entry proceduresEstablishing rescue procedures

Issuing entry permitsMonitoring confined spacesUsing personal protective equipment

Reviewing cancelled permitsSupervisor review of confined spaces


Supporting

Training & awareness- Training for entrants,

attendants, & supervisor

Documenting / record keeping- Permit retention- Training records- Written programme

Managing information- Training database

Audit Approach – Conducting Effective Audits


Conducting effective and efficient audits is dependent upon: • Utilising the team’s collective experience and knowledge of

the site operations, HSE standards, and auditing skills and techniques.

• Prioritising the topics to review in terms of importance/impact. • Basing your prioritisation on a thorough review and

assessment of management systems and controls. • Developing verification strategies to gather data that will

provide meaningful insights regarding compliance. • Communicating, communicating, communicating—both oral

and written communications that occur within the team and externally with affected parties are essential factors influencing the success of auditing efforts.

Basic Steps in the Typical Audit Process


Basic Steps in the Typical Audit Process – Overview


Step 1: Understand Management Systems

Conduct opening meetingConduct orientation tourReview audit strategyUnderstand details ofmanagement systems

Step 3: Gather Audit EvidenceEvaluate what needs to be doneDetermine depth & rigor of reviewSelect types of evidenceneeded & methods to gatherthemCompare practices againstrequirementsDocument results

Step 2: Assess strengths& Weaknesses

Consider potential impactsEvaluate management systemsSet priorities for verification

Step 4: Evaluate Audit ResultsEvaluate audit resultsWrite audit findings

Step 5: Report Audit FindingsConduct exit meeting

Prepare Draft Report

Obtain Review Comments fromCorporate HSELaw departmentFacility management

Issue Final Report toFacility managementOperationsCorporate HSELaw department

Develop Action PlanDevelop proposed action(s) toaddress each findingAssign responsibility for corrective actionDevelop timetable

Conduct Follow-upTrack status of correctiveactionsConfirm closure of findings

Pre-Audit Activities On-Site Activities Post-Audit Activities

Select & Schedule Facility Audit

Plan the Audit:Correspond with the facilityAssemble & distribute background informationAssign & communicateaudit responsibilitiesConduct pre-audit meeting

Select Team Members & Confirm their Availability



These are primarily the responsibility of the team leader, and have therefore been treated in detail later in this manual. It is important to remember that the team participates in the pre-audit preparation, by reviewing the background information to develop an initial understanding of the facility’s operations, modifying audit protocols as/if necessary, making their own travel arrangements, having liaised with the team leader on the time for the pre-audit team meeting, obtained and reviewed applicable regulations, and organising any materials or equipment necessary to perform the audit. Before the audit team arrives at the opening meeting, it should know enough about the site to be able to formulate some preliminary hypotheses about the major risks and HSE issues. There are several ways to accomplish this: • Auditors’ basic familiarity with company operations, policies,

and procedures • Pre-audit questionnaire • Other background material provided by the facility • Conversations between the team leader/members and plant

personnel • Review of applicable regulations In our experience, it is not uncommon for team members to begin their pre-audit preparation only on the plane that is taking them to the site. Even if this were adequate for team members, it is definitely not sufficient for team leaders.




Schedule auditSelect team membersContact & coordinate with thefacilityArrange for travelGather & distribute auditmaterials (reference documents, working paperpads)

Administrative

Assemble & review background information & applicable regulationsDevelop audit assignments & areas to focus onReview & discuss audit team responsibilities

Planning

GoalHit the ground running



The pre-audit activities provide the foundation upon which the team members build their understanding of the facility’s operations and establish an audit strategy. They also provide the facility with its first impressions of the audit programme and set the tone for audit team/facility interactions. The pre-audit activities should meet the following objectives: • Provide the audit team with sufficient plant information to

enable the team members to develop a basic understanding of the facility, the processes, and the HSE management systems employed. For example, the audit team should understand: − The type of facility being audited (e.g., chemical

manufacturing, injection molding, pulp and paper, distribution center, research laboratories, etc.).

− The employee population (e.g., 60-person plant or 1,000-

person plant, extent of contract employees and contractors used on site, business organisation).

− The general applicability of regulatory requirements to the

facility’s operations (e.g., presence of wastewater treatment plant, hazardous waste generator status, number of permitted air sources, applicability of industrial hygiene programmes [e.g., respiratory protection, hearing conservation, bloodborne pathogens, etc.], presence of a fire brigade and/or spill response team, availability of routine and emergency medical personnel, etc.).

• Inform the facility as to audit programme goals, objectives,

and procedures. Typically, the more information facility personnel receive prior to the audit, the less anxious and more comfortable they will feel about the on-site portion of the audit.



Using the information obtained, allow the audit team to develop a basic audit strategy prior to arrival at the site. In developing an audit strategy, the audit team should consider the following: • Audit objectives to be achieved, areas to receive emphasis,

and a preliminary division of responsibility among the team members.

• Questions or issues that need to be resolved during the preliminary meetings and points to be clarified.

• A preliminary agenda and schedule to be used during the audit.

To help facilitate the conduct of an efficient and thorough audit, the following pre-audit activities are frequently undertaken by the audit team leader and team members. 1. Corresponding With the Facility

Within the designated time frame established by the audit programme, the team leader should contact the facility to confirm the exact dates of the planned audit and to address the following: • Audit process and activities • Types of documents to be reviewed:

− Pre-audit information request − Pre-audit questionnaire

• Planning/logistical details:

− Safety and security requirements − Administrative/logistical details − Initial interview schedule



Audit Process and Activities In general, the team leader should explain the audit process and the types of activities to be undertaken while on site. The audit programme objectives, purpose, and scope, and the types of interviews and tours that will be conducted, should be explained. In addition, the team leader typically requests that the facility prepare a short presentation describing the plant operations for the opening meeting. The facility should have a general understanding of how the audit will proceed and what type of time commitment will be required for the various levels of staff affected by the audit. Types of Documents to be Reviewed It is important to determine the background information to be requested and reviewed before arriving at the facility. A protocol represents a plan to be used by the auditors in conducting an audit. Protocols are produced in advance, for each of the issues to be audited, sometimes annotated with relevant regulatory standards references.The purpose is to provide the audit team with a step-by-step guide to collecting evidence about a facility’s programmes and practices included within the scope of the audit. In addition, the protocol identifies selected topics and requirements included in the audit and provides guidance regarding how the team may audit or review against those requirements. However, it typically does not include all applicable performance requirements that the team may need to review a facility’s compliance status. Typically, audit team members review applicable performance requirements prior to the audit to determine whether any other requirements are appropriate for in-depth review and verification. Other purposes of an audit protocol include: • Tool for audit planning.



• Record for audit procedures—planned and completed. • Outline for working papers. • Record of changes in audit scope, procedures, etc., and the

rationale for any changes. • Basis for reviewing/critiquing an individual audit. Pre-audit information request. There are some materials that will be essential to have prior to the audit while other materials will only need to be looked at while on site. For some materials (e.g., training records, inspection logs, material safety data sheets, etc.), it is in the best interest of the audit team to see this material during the on-site visit. The team leader would be better advised to ask the plant to leave this information in existing files until the audit. As a result, the audit team will be able to see actual recordkeeping conditions and practices. The tables on the following pages list the types of information that are typically requested prior to the audit and information that should be available on site for review during the audit. The information listed in these tables serves only as an example. The type of documentation that should be requested will depend upon the staffing resources utilised (e.g., full-time corporate auditors, corporate or facility personnel with other full-time responsibilities, third-party auditors, etc.) and the time these resources realistically have for pre-audit preparation. When conducting pre-audit activities, it is important to keep in mind that the background materials should be requested early enough to ensure that there is enough time for: • The facility to assemble and send the information to the team

leader. • The team members to receive and review the materials prior

to the audit.



Audit Information to be Sent in Advance

Facility plot plan or map

Directions to the facility

Visitor safety requirements (e.g., personal protective equipment, orientation, specialised training, special clearances, etc.)

Completed pre-audit questionnaire

Description of the facility’s operations/processes

Facility organisation chart, showing HSE responsibilities

Local laws, regulations, and ordinances related to the scope of the audit

List of current environmental licenses, certificates, and authorisations

Copies of permits for wastewater discharges and example air emission permits

Recent regulatory agency inspection/enforcement correspondence

Recent internal and intra-company environmental, health, and safety audit reports

Table of contents for facility-specific environmental, health, and safety policies and procedures



Audit Materials to be Reviewed Upon Arrival On Site

Management objectives

Copies of all current environmental licenses, certificates, registrations, authorisations, and applications for such (including air and wastewater discharges; hazardous waste treatment, storage, or disposal activities; underground storage tanks; drinking water supplies; etc.)

Facility procedures and programme manuals (e.g., spill prevention plan, hazardous waste management contingency plan, respiratory protection plan, exposure control plan, hazard communication plan, etc.)

Effluent and emission monitoring reports

Training records

Hazardous waste manifests

Material safety data sheets (MSDSs)

Inventory of chemicals, including oils, in use or stored on site

Purchase orders for chemicals

Annual PCB reports (for the five years preceding the audit)

PCB transformers inspection records

Monitoring instrument calibration records and maintenance logs

Records of safety inspections, including reports of loss prevention surveys by insurance underwriters

First aid/dispensary records

Records of exposure monitoring and results, respirator fit testing, audiograms, etc.



Pre-audit questionnaire. In order to obtain a quick and concise facility profile to provide the audit team members with a basic understanding of the facility’s operations, a pre-audit questionnaire is typically sent to be filled out and returned by the facility. The pre-audit questionnaire should be a tool that is easily processed by the facility and provides the auditors with basic information enabling them to begin planning for the audit. An example of a pre-audit questionnaire is provided in Appendix A of this manual. Although the pre-audit questionnaire may provide the audit team members with a basis to begin their understanding, the auditors should critically review the information and remember that the information provided may not be completely reflective of the facility’s operations. For example, the facility may indicate on the pre-audit questionnaire that it does not have underground storage tanks on site. However, the facility may be unaware that there are two abandoned underground storage tanks that were inadvertently omitted from the facility’s initial spill prevention plan and have long been forgotten. During the on-site phase, the auditor may need to change his/her original audit strategy (based on the pre-audit information) to ensure that the protocol areas assigned are properly addressed. Planning/Logistical Details During the team leader’s communications with the facility contact, any planning details should be resolved, such as: • Safety and security requirements • Administrative/logistical details • Initial interview schedule



Safety and security requirements. The team leader should clearly understand the personal protective equipment requirements for visitors who will be touring and inspecting any and all site areas (e.g., safety glasses, hard hats, safety shoes, safety clothing, etc.). The team leader should also obtain information regarding whether: • Safety orientations and specialised training (e.g., hydrogen

sulfide, respiratory protection, underground mine safety, etc.) are required to enter the general facility or regulated areas. The team leader should inquire as to the time needed to complete the necessary orientation or specialised training (e.g., 15 minutes, two hours, eight hours, etc.). Depending upon the training necessary, the team leader may need to adjust the audit schedule.

• Security clearances/passes are required for audit team

members or vehicles to enter the facility or regulated areas. • Restrictions apply to team members who are not citisens of

the country where the facility is located. • Escorts are required for team members touring and inspecting

facility areas. Administrative/logistical details. The team leader should cover the administrative and logistical details with the facility contact as early as possible in the pre-audit process. Administrative and logistical details that may need to be addressed include: • Requesting information regarding travel to the facility, as well

as lodging in the vicinity of the site. • Arranging for badges, clearances, car passes, safety

orientation/ training, and escorts. • Arranging for the audit team to stay after normal business

hours and observe second or third shift operations, where applicable.



• Scheduling the opening meeting and setting a tentative time for the closing meeting.

• Establishing a time during the day when the audit team can

meet daily with the key HSE staff to discuss their preliminary findings and concerns. These meetings are typically referred to as daily debriefs.

• Arranging for meals (working breakfasts, lunches, or dinners). • Requesting that the facility prepare a brief presentation on the

facility’s operations for the opening meeting. • Arranging for a meeting room to be available to the audit team

during its visit. • Informing facility personnel of any documentation/equipment

(e.g., paper copy of the facility’s presentation, telephone directory, employee rosters, telephone, overhead projector, slide projector, printer, fax machine, photocopy capabilities, etc.) that needs to be made available for the audit team.

Initial interview schedule. The team leader may want to obtain the names of facility contacts to begin scheduling interviews with key HSE staff for the first one or two days of the audit. The benefits associated with having the team leader and the facility put together an initial interview schedule with key HSE staff are two-fold: 1) the facility is given an opportunity to feel a part of the audit process and to establish interviews at convenient times, lessening the impact on their day-to-day activities; and 2) the audit team will be able to begin their understanding of how the facility manages environmental, health, and safety areas immediately after the opening meeting and orientation tour.



Finally, after arrangements have been agreed upon, the team leader should send a letter to the facility contact confirming those arrangements along with an outline of the audit process and activities, and the audit objectives, purpose, and scope. In addition to the pre-audit questionnaire, a list of the materials that the audit team would like sent prior to the audit is typically sent as an attachment to the confirmation letter. An example confirmation letter is provided in Appendix B. 2. Assembling and Distributing Background Information

Assembling Background Information Following the receipt of materials from the facility, the team leader should begin the task of assembling the available background information. This step, in general, will enable the team to develop an effective audit strategy tailored to an individual facility. This task typically involves: Reviewing facility information and responses to the pre-audit questionnaire and/or contacting the facility if background information has not been received within the specified time period. Contacting the facility to clarify any ambiguous or incomplete information received. Contacting the legal department, as appropriate, to ascertain whether the facility has any outstanding litigation or history of compliance problems. Obtaining relevant company policies and procedures and applicable federal/national, state/provincial, and local regulations. Identifying site-specific situations and requirements that may require modification of the standard audit protocols.



Assembling the appropriate audit protocols to be used for the audit, and assigning them to individual auditors according to their skills, experience and background. Distributing Background Information The background information collected by the team leader should be distributed in a timely fashion to allow the team members enough time to review the information prior to the audit. 3. Assigning and Communicating Audit Responsibilities

As the necessary background information is gathered and reviewed, the team leader should make an initial allocation of the functional areas (e.g., air pollution control, hazardous waste management, industrial hygiene, employee safety, etc.). This task involves matching the talent and expertise of the team members with specific tasks or protocols, as well as taking into consideration the team members’ prior audit assignments. These assignments are typically made by the audit team leader with input from the individual auditors. If the audit team is not in one geographical location, a conference call or video conference can be set up to establish and communicate the audit responsibilities. In addition to the assignment of functional areas, the team leader should clearly communicate all pertinent information and audit team responsibilities during the pre-audit phase. For example, audit team members are typically responsible for: • Reviewing the background information supplied by the team

leader to begin developing an initial understanding of the facility’s status with respect to each assigned functional area(s).

• Modifying the audit protocols, as appropriate, to incorporate

state/provincial, local, or facility-specific requirements or special facility conditions, plans, procedures, etc.



• Making travel/lodging plans. • Obtaining and reviewing the applicable federal/national, state/

provincial, and local regulations. • Bringing the necessary audit materials/equipment for the audit

as directed by the team leader (e.g., personal protective equipment/ clothing, background information, audit protocols, working paper pads, regulations, computer, etc.).

4. Conducting Pre-Audit Meeting

A pre-audit meeting may be the last step of the pre-audit planning phase. This meeting is typically held immediately before the audit (i.e., evening or early morning prior to the audit) if team members are not located in the same geographical area. The purpose of the pre-audit planning meeting is to clarify any details regarding the protocols and to develop an overall audit strategy. An audit strategy is essentially an outline of the tasks that may need to be done in order to complete the five-step on-site audit process, how each task is to be accomplished, and the time required to complete each step. Audit protocols serve as a basis for developing this strategy. During this meeting, team members also: • Discuss and evaluate background information received from

the facility and determine if there are any overlapping areas (e.g., industrial hygiene and employee safety regarding personal protective equipment).

• Identify protocol steps and questions that have been modified

to reflect special facility conditions or unique state/provincial or local regulatory requirements.



• Confirm that they understand how much time to allocate during the on-site phase to complete the five-step audit process. Typically, each auditor should allocate his/her time on site as indicated below.

Allocation of Time On Site

Percentage of Time Activity

15 Understand the management system for assigned topics

10 Assess the apparent strengths and weaknesses of those management systems

40 Gather audit evidence 30 Re-assess strengths and weaknesses and evaluate audit results 5 Formally report the audit findings to site management

In addition, some audit programmes find it beneficial, whenever possible, to conduct a pre-audit visit. During this pre-audit visit, the audit team leader will have a one-day meeting with facility personnel to review the audit programme objectives, scope, and approach; establish a preliminary interview schedule; and/or tour the facility to better understand the operations. The results from this pre-audit visit are shared with the audit team members prior to the audit to assist in the development of an audit strategy.

On-Site Opening Activities


On-Site Opening Activities – A “Typical” Audit


Daily Debrief

8:00 - 8.30

Daily Debrief

8:00 - 8.30

Daily Debrief

8:00 - 8.30

Meet with HSE Staff8:00 - 8.30

Opening Meeting

11:00 - 12.00

Understand Details of Systems

8:30 - 12.00

Gather Audit Evidence

8:30 - 12.00

Continue to Gather Audit Evidence

8:30 - 12.00

Wrap Up Loose Ends

8:30 - 11.00

Conduct Close-Out Meeting

11:00 - 12.00

Lunch Lunch Lunch Lunch Lunch

Orientation Tour

1:00 - 2.00

Review Audit Plan

2:00 - 2.30

Assess Strengths and Weaknesses

1:00 - 2.00


12:30 - 2.00

Gather Audit Evidence

2:00 - 4.30


12:30 - 4.00

Team Meeting

4:30 - 5.30

Team Meeting

4:00 - 5.30

Understand Details of Systems

2:30 - 5.30

Evaluate Audit Results

2:00 - 6.00??

Travel to Site

Travel Home

Team Activities

Monday Tuesday Wednesday Thursday Friday

Individual Activities

On-Site Opening Activities - Purpose of the On-Site Opening Activities


The opening meeting and the orientation tour are the initial activities undertaken by the audit team to begin their understanding of the facility’s management systems.

On-Site Activities

Opening meeting

Interviews

Opening meeting

Interviews

Key plans, procedures,policies

Document Review

Key plans, procedures,policies

Document Review

GoalUnderstand, who, what, where, how

Orientation tour

Observation

Orientation tour

Observation

The on-site opening activities are intended to provide the audit team with a broad and general overview of facility operations and issues. The audit team members typically obtain this broad overview by: • Conducting an opening meeting with facility management

upon arrival at the site to discuss overall facility operations and the organisational structure used to help facilitate the implementation of compliance activities.

• Conducting an orientation tour with key facility personnel to

obtain a general orientation to the plant, including its layout and size, location of operations, and location of those activities pertaining to the audit scope.

• Reviewing the audit strategy as a team to ensure that the

audit scope includes all the applicable audit topics and that resources are allocated appropriately, based upon the information gathered thus far, for each of the audit topics under review.

On-Site Opening Activities - Conducting the Opening Meeting


The purpose of the opening meeting is to: • Describe to facility personnel the overall objectives of the

audit programme, and the purpose, scope, and approach of the audit.

• Gain an initial overview of the facility’s programmes and

practices established to manage environmental, health, and safety issues relevant to the scope of the audit.

The opening meeting will, to a large extent, influence the overall outcome of the audit; therefore, it is important that this meeting be conducted in a professional manner which allows a comfortable exchange of information between the audit team and facility personnel.

The following table outlines the typical activities and topics included in the opening overviews of the: • Audit process presented by the team leader. • Site operations, programmes, and procedures presented by a

facility representative.



Key Activities in Presenting the Overview of the Audit Process

Meet with the facility manager and key facility personnel

Have the team leader open the meeting

Use an opening meeting discussion guide

Use an overhead projector with viewgraphs, as appropriate

Encourage discussion among meeting participants

Tentatively schedule the exit meeting or reconfirm the previously established schedule

Reconfirm the time for the daily debriefs

Topics Typically Addressed in the Facility’s Overview

Production operations, capacities, raw materials, and product lines

Operating hours (staff and operations personnel)

Employee profile (number of salaried and hourly staff, union status)

Overview of HSE programmes

Any major HSE issues/problems

Community issues

Government inspections and compliance history

Regulatory climate



Before beginning the orientation tour, the team leader should also reconfirm the following logistical and administrative arrangements discussed during the pre-audit phase:

Necessary clearances Safety rules and procedures Work space for the team Names of facility contacts

On-Site Opening Activities - Conducting the Orientation Tour


The purpose of the orientation tour is to provide the audit team a general familiarity with the plant layout and operations. Typically, this tour takes one to two hours. During the tour, the team members should: • Stay together. • Focus on obtaining an

overview, not on making inspections.

• Carry and annotate, as

necessary, a site plot plan. • Take notes on areas the

auditor wishes to revisit and employees/ staff that he/she may need to interview.

On-Site Opening Activities - Reviewing the Audit Strategy


After the team has gained a general understanding of the facility’s operations and issues, they should meet to review the audit strategy and discuss the next steps. For instance, the audit team should finalise an interview schedule for the remainder of the day or for the next day. Often, the team may need to sit down with the facility HSE coordinator(s) to clarify HSE roles and responsibilities as well as to gather additional information regarding the management systems in general and the overall applicability of performance requirements, if not already addressed in the pre-audit phase.

Team Member

Team Member

Team Member

Team Member

Team Member

7:00 am

8:00 am

9:00 am

10:00 am

11:00 am

12:00 pm

1:00 pm

2:00 pm

3:00 pm

4:00 pm

Breakfast with TeamBreakfast with Team

LunchLunch

Daily Debrief with Facility StaffDaily Debrief with Facility Staff

Time

On-Site Opening Activities - Example of an Opening Meeting Discussion


ChemplantOpening Meeting Discussion Guide

Audit Team: NPC Contacts:Lead Auditor: Plant Manager:John Clarke Jerry OsborneAuditor: Safety and Health Supervisor:Peter Tillson Pat DawsonAuditor: Environmental Co-ordinator:Paula Brown Chris Carson

Local Address: Facility Address:Best East Inn 875 Willow StreetAny road Anytown, AnyplaceAnytown, Anyplace (222) 222-2222(222) 333-3333



Purpose

The purpose of this audit is to conduct a comprehensive review of the environmental, health, and safety activities in order to: • Verify compliance with applicable national, regional (local,

state etc.), and local environmental, health, and safety laws and standards

• Verify conformance with corporate, company, and facility

environmental, health, and safety policies and procedures • Determine whether activities are consistent with good

environmental, health, and safety management practices and whether systems are in place and functioning

Scope

This environmental, health, and safety audit will address the facility’s compliance and management systems in the following areas: • Water Pollution Control, including Spill Prevention and Control • Air Pollution Control • Solid and Hazardous Waste Management • Employee Safety • Industrial Hygiene • Loss Prevention



Approach

The audit is based on: • A physical survey of the facility • Examination of a sample of environmental, health, and safety

administrative, technical, and operating records available at the facility

• Interviews and discussions with key facility management and

staff • Verification procedures designed to examine the facility’s

application of and adherence to environmental, health, and safety laws and regulations

Period of Review

January 3, 2000 through the last day of the audit Reporting

A hierarchical reporting scheme will be used: Who What When Facility/HSE Supervision All deficiencies noted When noted

Facility Manager All deficiencies noted Periodic, exit interview, final report



Report Schedule

A draft report will be issued within three weeks of the close-out meeting. After receiving comments on the draft report, a final report will be issued.

Understanding HSE Management Systems


Understanding HSE Management Systems - Why Understand HSE Management Systems?


















Understanding HSE Management Systems - Why Understand HSE Management Systems?


Purpose

The first step of the on-site activities is to develop an understanding of how the facility manages its HSE activities so that the audit team can: • Gather information that will be used to help set priorities

among the audit topics to review. • Gain insights regarding how effectively and efficiently an EHS

topic is being managed and, thus, establish a context for evaluating the audit results.

• Identify potential underlying causes that contribute to

compliance-related deficiencies. Other Driving Forces

In addition, audit standards—such as the ICC Charter for Sustainable Development, ISO 14000, EMAS, and BS7750—require a thoughtful review and evaluation of the facility’s systems for managing HSE obligations. Our approach to Step 1: Understand Management Systems will be to: • Explain what HSE management systems are and some of the

specific activities involved. • Describe how the auditor should go about understanding HSE

management systems in Step 1 of the audit process.

Understanding HSE Management Systems - What Are Management Systems?


In the simplest sense, HSE management systems are the actual processes used by a facility to achieve and maintain conformance with established standards, including programmes, policies, equipment, administrative controls, etc. The auditor’s role is to find out how the facility really manages its HSE obligations.

Source: Systemation, January 15, 1959, published by Systemation, Inc.,

Colorado Springs, Colorado.



With the onset of emerging international environmental management standards, HSE management systems can be described in terms of key processes that are aligned with an appropriate organisation and resources.

Organisation Resources

Supporting


“How are we doing?”“What do we needto manage?”

“How shouldwe manage it?” “Let’s manage it!”

Training and awareness Documenting/recordkeeping Managing information

Typical HSE Management Processes

Foundation



The key processes in management systems can be defined and categorised as follows:

Basic Step On-Site Activities Outcome

Assessing is used by the facility for identifying conditions / aspects & materials on facility that have HSE implications, & for determining the applicability of regulations

Identify / evaluate HSE issues Identify on-site hazards Review HSE activities and projects Review applicable regulations

Planning is used by the facility for designing and establishing programs & systems for HSE compliance

Formulate HSE strategies & policies with clear objectives & targets that reflect the importance of the HSE issues applicable on site

Develop HSE procedures for compliance & record keeping activities, as well as prevention plans with specific operating criteria

Identify & design engineered controls & equipment Develop emergency response procedures; create

procedures regarding critical HSE activities / issues or departures from established criteria

Implementing is used by the facility for ensuring effective and consistent implementation of its HSE programs

Acquire permits Disseminate policies & procedures Assign & communicate roles & responsibilities Install, calibrate & maintain engineered controls Handle situations that deviate from an established

standard Undertake activities in accordance with established

schedules Reviewing is used by the facility for measuring &

assuring HSE program effectiveness Conduct drills Conduct inspections / self-audits in accordance with

documented procedures Review compliance data / performance Track continuous improvement Review the effectiveness of management systems Undertake corrective actions in response to identified

directors from procedures or established criteria

Assessing“What do we need tomanage?”

Planning“How should we manageit?”

Implementing“Let’s manage it”

Reviewing“How are we doing?”



Supporting processes (training and awareness, documenting/ recordkeeping, and managing information) tend to cross over the four key categories of assessing, planning, implementing, and reviewing.

Training and Awareness

Training programmes, both formal (e.g., classroom) and informal (e.g., on-the-job), in-house and external, for ensuring that each HSE programme element is understood and implemented properly. Programmes and/or other means for communicating HSE procedures, and addressing critical HSE issues and potential consequences of departure from specified operating procedures. Competent staff performing HSE work.

Documenting/ Recordkeeping

Systems for legibly documenting and dating activities (e.g., routine reports, inspections, audits, training, etc.) conducted during the implementation of the HSE programmes. Systems for retaining documentation in an accessible and orderly manner for periods of time required by regulation or internal standards and for removing obsolete documents.

Managing Information

Systems for communicating relevant information across the various levels and functions of the organisation. Mechanisms to track and evaluate compliance information. Systems for responding to relevant communications from external parties.

Organisation and resources should be aligned to support effective HSE management systems.

Organisation

Clear assignments and understanding of HSE responsibilities and accountabilities among HSE and line management. Visible HSE commitment and support by management. Appropriate high-level HSE reporting.

Resources

Sufficient number of qualified HSE staff. Adequate HSE staff/responsibilities to cover all business/organisation groups within the facility. Availability of needed financial and technological resources.

Understanding HSE Management Systems - The Methodology for Understanding HSE Management Systems


There are three principal approaches used to understand HSE management systems within the context of an audit.

Look at Key Equipment/Facilities Review Key Programme Documents

Talk to Key People

Contains (or manufactured with, if applicable)Contains (or manufactured with, if applicable)(insert name of substance), a substance which har ms(insert name of substance), a substance which har ms

public health and environment by destroying ozonepublic health and environment by destroying ozonein the upper atmosphere in the upper atmosphere

WarningWarningContains (or manufactured with, if applicable)Contains (or manufactured with, if applicable)(insert name of substance), a substance which har ms(insert name of substance), a substance which har ms




WarningWarning

Contains (or manufactured with, if applicable)Contains (or manufactured with, if applicable)(insert name of substance), a substance which har ms(insert name of substance), a substance which har ms






WarningWarning

0102030405060

0102030405060



Technique Activities

Talk to Key People

Talk to several people (e.g., line management, HSE staff, operating personnel, maintenance personnel) to obtain a comprehensive understanding of the activities that are in place to manage compliance and to increase your sense of confidence in the information obtained. Summarise the information obtained from each interviewee to verify the completeness of your understanding. Probe to understand inconsistencies in the information obtained. Examples of what auditors should endeavor to understand during interviews include:

• What is meant by the scope of the facility’s programmes? For example, when the facility says that, “All employees receive hazard communication training,” does this mean all employees, all employees who work in certain areas, or something even narrower?

• How does the facility handle seasonal, situational, or “non-normal” activities? For example, how does a particular activity work on the off-shift, or when a key person is on vacation, etc.?

• How does the facility develop data for preparing compliance-related reports, for determining compliance, or for identifying HSE-related problems? To understand this, the auditor may, for example, want facility personnel to describe or demonstrate how they reconcile the monthly inventory for an aboveground storage tank

Look at Key Equipment/Facilities

Walk around the facility to understand the nature of the HSE issues that need to be managed and the types of engineered controls in place. (This is primarily accomplished during the orientation tour.) At this stage, auditors should focus on understanding the nature and rationale of the engineered controls used to manage HSE hazards.



Technique Activities

Review Key Programme Documents

Briefly review procedures, plans, etc., that explain how the facility manages HSE obligations. For example: • Compliance-related programme documents • Operational procedures • Checklists or inspection forms • Training programme description At this stage, auditors should focus on understanding: • How the programme is supposed to work. • Tools used by the facility for ensuring that the programme

works as designed and is effective. There are some key questions auditors should try to answer when understanding each stage of the management process.



Management Process

Things the Auditor Wants to Learn

Assessing • How are regulations tracked (i.e., identifying, tracking, interpreting, and communicating regulations)?

• How are HSE risks and effects assessed (e.g., waste stream inventory, air emissions inventory, natural resources consumed, likelihood and magnitude of unplanned events, potential exposure to hazardous substances, etc.)?

• How does the facility manage changes in procedures or facility design (e.g., HSE review and consideration for new products, processes, equipment, acquisitions and divestitures, maintenance modifications, etc.)?

Planning • What type of basic compliance programmes (e.g., permitting, monitoring, training, recordkeeping, reporting, etc.) have been or are being established and do they include critical operating parameters and schedules?

• How does the facility prepare for emergencies (e.g., developing scenarios, response capability, response plans, etc.)?

• What type of issue-specific risk reduction programmes (e.g., groundwater monitoring, pollution prevention, waste management practices, spill containment programmes, ergonomics, engineered controls, etc.) are developed?

• What engineered controls or alarms are in place to help achieve desired results?

• What measures have been taken to reduce the likelihood of nonconformance with established criteria?

• What types of policy and related goals and objectives are established (e.g., vision statements, basic policies and guiding principles, specific goals and milestones, etc.)?

• What strategies are developed for managing HSE risks and effects?



Management Process


Implementing • Has the facility fully implemented the various HSE programmes?

• How does the facility maintain operating equipment (e.g., preventive maintenance programmes, testing and monitoring, etc.)?

• How is nonconformance with established criteria handled?

Reviewing • Are facility operations being inspected (e.g., routine walk-throughs, use of checklists, etc.)?

• How are HSE effects being measured? • Does the facility analyse its performance (e.g., evaluate

“findings,” “lessons learned,” trends, etc.)? • Are programmes in place for developing, implementing,

and tracking corrective actions? Training and Awareness

• What formal training have key personnel had to assist them in performing their HSE tasks and functions?

• What training and awareness activities are conducted to provide an understanding of HSE obligations and responsibilities?

• What type of training programmes (e.g., compliance-related training, emergency drills, on-the-job training, etc.) are available?

• What types of experience or background are required to perform HSE tasks?

Documenting and Record-

keeping

• Are procedures and practices for both compliance and remediation generally written down?

• What records are routinely developed and retained in carrying out various tasks and functions?

• What exception reports are developed? • What is the general nature or character of the

documentation that is developed? • How does the relative importance of HSE activities

correspond to the nature and level of documentation that is developed?

• Where is information retained? • How long is information retained?



Management Process


Managing Information

• How is important HSE information conveyed to personnel?

• Can the facility readily access HSE information? • How does the facility ensure that reports are submitted

in a timely manner and that essential records are retained?

• Are there procedures in place for responding to external requests for information?

Organisation • How are responsibilities and accountabilities defined, established, and communicated?

• How are assignments of responsibilities reinforced? • Are any key responsibilities overlapping, shared, or

conflicting? • What potential exists for “conflict of interest” in

accomplishing key HSE tasks and functions? • How is authority granted to carry out assigned

responsibilities? Resources • How have responsibilities for implementation been

communicated to personnel who need to know? • Are there sufficient resources to carry out the various

HSE programmes? • Who has the authority to waive adherence to, or

conformance with, an established standard or requirement and are deviations recorded?

• Are there perfunctory approvals—authorisations without understanding what is involved?

Understanding HSE Management Systems - How Much is Enough?


Auditors frequently wonder whether they have collected enough information and the right kind of information to substantiate their understanding of a facility’s key management systems/programmes and physical controls. Listed below are tips for determining how much is enough. You have probably gathered enough information if:

You understand both system design (e.g., facility policies and procedures regarding regulatory tracking) and implementation (e.g., availability of the latest regulations on site).

You have interviewed all of the key personnel involved in key

HSE functions or tasks, and you can summarise to their satisfaction the basic programmes, practices, and control systems.

You understand the probable cause(s) of any differences

between management’s and employees’ perspectives, or between environmental and operational personnel’s perspectives.

You understand the types of activities that are applicable and

the range of activities being managed by the facility.

You understand the roles and responsibilities of HSE staff, the mechanisms to share relevant information, and the processes for retaining information in an accessible manner.

Once the auditor has developed a basic understanding of the management systems associated with assigned audit topics, he/she is now ready to move on to Step 2 and assess the systems under review.

Exercise 1A Understanding HSE Management Systems - Confined Space Entry


Objective

The purpose of this exercise is to demonstrate how the auditor obtains information to understand the management systems in place at a facility (Step 1 of the audit process). Background

Based upon the preliminary information provided to you by the facility, you know the following: • The facility has confined spaces, which are entered by facility

personnel and contractors. • The facility has a confined space entry programme along with

permits. • The facility is a chemical manufacturer that handles a variety

of toxic and flammable substances. Instructions

Describe the steps you would take to understand the management systems used by the facility to implement the confined space entry programme. Include in your description: • Whom you would talk to. • What questions you would ask. • What documents and physical facilities you would want to look

at. • Any other activities you would conduct while completing Step

1 of the audit process for this topic.

Exercise 1A Understanding HSE Management Systems - Confined Space Entry


Whom would you talk to? What questions would you ask? What documents and physical facilities would you look at? Other activities?

Exercise 1A – Potential Answers Understanding HSE Management Systems - Confined Space Entry


Whom would you talk to? HSE Manager Maintenance Manager Maintenance Staff What questions would you ask? What is the procedure for confined space entry? Where are they formalised (HSE manual)? To Maintenance Staff – What do they do before and while entering confined space? Training PPEs? What documents and physical facilities would you look at? Permit to work Inventory of confined spaces Other activities? Ask to go and see the confined spaces Ask to observe if any maintenance is scheduled during the audit

Exercise 1B Understanding HSE Management Systems - Spill Control


Objective

The purpose of this exercise is to demonstrate how the auditor obtains information to understand the management systems in place at a facility (Step 1 of the audit process). Background

Based upon the preliminary information provided to you by the facility, you know the following: • The facility has aboveground storage tanks containing

petroleum and chemical products. • The facility has tank truck loading/unloading operations. • The facility has a spill control team that has been provided

training. • A Spill Prevention Control and Countermeasures plan is on

site. Instructions

Describe the steps you would take to understand the management systems used by the facility to implement the spill control programme. Include in your description: • Whom you would talk to. • What questions you would ask. • What documents and physical facilities would you want to look

at. • Any other activities you would conduct while completing Step

1 of the audit process for this topic.

Exercise 1B Understanding HSE Management Systems - Spill Control


Whom would you talk to? What questions would you ask? What documents and physical facilities would you look at? Other activities?

Exercise 1B – Potential Answers Understanding HSE Management Systems - Spill Control


Whom would you talk to? HSE Manager Maintenance Manager and staff who supervise loading/unloading Spill control team What questions would you ask? What is the procedure for spill control and countermeasure? Who is in charge? What training is carried out? Is external personnel (truck drivers) trained/supervised? What happens if a spill occurs on a Sunday night? What documents and physical facilities would you look at? Spill prevention control and counter measure plan (Accident/Incident/Near misses register) – Loading/unloading, secondary containment Spill control kits – Waste registers Hazard material inventory – Quantities and storage locations Other activities? Observe loading/unloading

Effective Interviewing


Effective Interviewing


Interviewing is one of the primary techniques used in gathering audit information. Interviews provide auditors with: • A time-efficient way of gathering both broad general

information and specific details from people who should know. • A means to confirm hypotheses about site conditions,

changes, needs, and opportunities. • A current and credible source of facts and perceptions that

complement written information and physical observations. Thus, good interviewing skills are essential to the successful completion of the audit. While in one sense interviewing is a skill that comes naturally to most people, good interviewing techniques, which emphasise interaction between interviewer and interviewee, must be developed. By remembering some basic elements of good interviewing skills, the interviewer not only will be successful in gathering the information he/she desires, but will also find the interview process much more pleasant.

Effective Interviewing – The Basic Interview Process


A significant portion of the auditor’s field time is spent asking questions of and engaging in discussions or conducting interviews with facility staff. While the setting, duration, and degree of formality of such interviews can vary, all audit interviews follow this common pattern.

Planning

Opening

Conducting

Closing

Documenting

1. Planning the Interview

Prior to conducting the interview, the auditor should identify personnel to be interviewed, outline the objectives to be accomplished, and plan how to maximise the effectiveness of the interview. Key considerations include: • Iron out logistics—set a specific time and place for the

interview. • Define the desired outcome—identify the types of information

desired and/or areas to be addressed. The types of information gathered during interviews can be characterised as “hard” or “soft.”



Hard • Quantitative data regarding emissions, exposure monitoring, etc.

• Records prepared for compliance purposes • Historical information about a site

Soft • Why did something (not) happen? • How does the process/system work? • How does the work really get done?

Most HSE audit interviews mix “hard” and “soft” information needs. • Organise your thoughts—develop a logical sequence of

questions. • Be prepared—HSE interviews take place under all types of

distracting conditions, so plan appropriately.

2. Opening the Discussion

The quality of information gathered during an interview is closely related to the interviewee’s sense of comfort. The level of openness that develops during an interview, along with the interviewee’s confidence in the topic being discussed, depends a great deal on the rapport and atmosphere established during the initial contact.



In constructing an atmosphere where positive rapport can be established, auditors should follow these general guidelines: • Arrive on time. • Introduce yourself. • Ensure appropriateness of time. • Explain the purpose of the discussion. • Explain how the information gathered in the discussion will be

used. In addition, auditors should strive to build the desired sense of comfort and confidence by: • Cultivating a friendly, nonthreatening discussion. • Attempting to respond to the interviewee’s social style. For

example:

Social Style Orientation Example Interviewee

Questions/Responses Analytical/ Thinker

Technically-oriented

“What’s your methodology?” “How will the results be used?”

Driving/Doer Results-oriented

“What can I do for you in the next ten minutes?” “What is the outcome of all this time spent talking with auditors?”

Amiable/ Feeler

Relationship-oriented

“How are you enjoying our plant/town?” “Why are we having this interview?”

Expressive/ Intuitionist

Social recognition-oriented

“Did you notice how effectively our waste management process operates?” “We believe we’ve made the most improvements in this area of anyone in the industry.”



• Refraining from portraying a condescending, arrogant, know-it-all attitude.

• Acting in a supportive and nonjudgmental manner. • Ensuring that the auditor and the interviewee are “on equal

ground.” 3. Conducting the Interview

Style, Flow, and Tone Once a comfortable interview setting and rapport have been established, the auditor should focus toward obtaining specific information from the interviewee. Some examples of specific items that an auditor should address include: • Request a brief overview of the interviewee’s job. • Discuss the interviewee’s responsibilities in relation to the

topic(s) being reviewed. • Use language that the interviewee can understand. • Start with some general questions, then gather more detailed

information.

• Resolve ambiguities through constructive probing. • Do not exceed the agreed-upon time limit without first

obtaining the interviewee’s approval. • Provide feedback as appropriate.



Establishing an appropriate interview style, flow, and tone is essential to the interview process. For example: Style • Avoid interrogation

• Be empathetic • Avoid defensiveness • Be calm, objective, and non-partisan • Be courteous, alert, and responsive

Flow • Avoid disjointed transitions that damage rapport (e.g., cutting the interviewee off because you are trying to transition to a new topic)

• Use word association to change topic focus (e.g., if the interviewee mentions his/her training, use this opportunity to raise any training questions)

Tone • Be genuine and take an interest in the interviewee’s responses • Use a soft, friendly voice

Types of Interview Questions Appropriate questioning should be utilised to obtain the desired information. For example: Type of Question

Example Questions Typical Response Outcome

Relative Value of Response Information to Auditor

Leading “Of course you notify the state of a planned discharge.” “You do test the wells every month, don’t you?”

Often unintentionally “lead” the interviewee to the desired answer.

−

Yes/No “Do you have a spill response procedure?” “Have you conducted waste audits?”

Usually receive only a “yes” or “no” answer. +

Close-Ended

“What is your current production capacity?” “What is the capacity of your wastewater treatment plant?”

Usually receive a one-word answer. +



Type of Question

Example Questions Typical Response Outcome

Relative Value of Response Information to Auditor

Open-Ended

“How are new chemicals selected for use in your process?” “What analyses are performed to assess HSE risks and liabilities?”

May prompt a more detailed response.

+++

How Do You Know

“How does your facility ensure that workers receive appropriate HSE training?” “How does your facility ensure that PCB transformer inspections are performed?”

Can provide the most insight into how things are actually managed. +++

Active Listening Another important component to conducting successful interviews involves active listening. Active listening allows the interviewer to: • Summarise information accurately. • Test the interviewee’s understanding of the topic being

discussed. • Probe for confirmation. • Facilitate the interview. • Display empathy/establish greater rapport. All auditors should develop the following listening techniques: • Wait until the current question is answered before asking

another question. • Encourage the interviewee in a nonverbal manner (e.g.,

maintain eye contact, display attentiveness, etc.).



• Interrupt only if you sense avoidance—people often feel more secure when given an opportunity to speak without being interrupted.

• Imagine the interviewee’s situation (i.e., put yourself in the

interviewee’s job scenario). • Listen for emotions and attitudes as well as facts. In addition, the interviewer should remember to listen over 90 percent of the total interview time. Paraphrasing/Summarising Information Learned The technique of paraphrasing can aid the auditor in confirming or clarifying something said or implied by the interviewee. There are three levels of paraphrasing: Level Accomplishment Example Paraphrase

1 Confirms or clarifies expressed thoughts and feelings

“So there are three factors that determine the present situation...”

2 Confirms or clarifies implied thoughts and feelings

“You would really like to change this situation...”

3 Surfaces core thoughts and feelings

“You are concerned that your company’s engineering approach is outdated...”

In using this technique, it is important to: • Paraphrase completely. • Match the levels of emotional intensity and factual content. • Use levels 1 and 2 freely; treat level 3 with caution.



Nonverbal Communication A significant portion of the information exchanged in any interview is done nonverbally. Nonverbal communication is a combination of the meanings expressed by the interviewer and the interviewee through gestures, facial expressions, voice inflections, and posture. Auditors should be aware of nonverbal communication and pay attention to the following: • Shake hands. • Maintain eye contact. • Keep the right distance. • Tolerate silence. • Mirror the interviewee’s body movements. • Be sensitive to culture and customs (especially outside of your

native country). Actions to Avoid In addition to the positive actions that should be taken in preparing for and conducting the interview, there are also some actions the interviewer should avoid. These include: • Debating with the interviewee or wasting time disagreeing on

any one point. • Forcing a meeting with the interviewee if he/she is otherwise

occupied. • Rushing the interview. • Amplifying criticism given. • Using sarcasm or subtle humor. • Telegraphing your assessment of the interview. • Jumping to conclusions/not watching your assumptions. • Communicating incomplete or unsubstantiated findings or

conclusions. • Using a tape recorder.



4. Closing the Interview

The closing phase of the interview should appear to reach a natural conclusion. In closing the interview, the auditor should strive to: • Pace the interview to avoid rushing and, if necessary,

schedule additional interview time when mutually convenient. • Summarise and confirm information learned. • Try to bring closure to sensitive issues/questions raised earlier

in interview. • Never exceed the allotted time without confirming with the

interviewee. • Ensure that the interviewee is psychologically ready to leave

the interview (e.g., not unduly worried, upset, etc.). • End with an “open door” for communication in either direction. • End on a positive note—thank the interviewee for his/her time

and cooperation. Because the interviewee may feel that this phase of the interview is his/her final opportunity to be heard, closing questions used by the auditor can be extremely productive. Some examples include: • Is there anything you expected me to ask about that I did not

mention? • A little earlier you said that... • We have heard that...Is that really the way it is? • Can you elaborate on that point? • Would others say the same? • Is it like that at other times/places/divisions/companies? • Playing devil’s advocate, would some people fundamentally

disagree? These questions clarify open issues and help to resolve contradictory statements.



5. Documenting the Interview

Most of us have limited memories, especially when it comes to recalling what people have told us; if we have no record of an interview and have to rely entirely on our memories, we are bound to make mistakes and omit important points. Therefore, in audit interviews, it is safer to take notes and not to rely on an imperfect memory, especially when you are interviewing several people during the course of any given day, which is most often the case during audits. The only problem with taking working paper notes is how to do so without distracting or intimidating the interviewee. The break in eye contact that occurs during the writing process can be distracting for the non-writing individual. Similarly, not knowing what or why an auditor is writing may intimidate the interviewee. With these points in mind, review the following suggestions on how to take notes during interviews: • Never try to hide the fact that you are taking notes. Rather,

draw the interviewee’s attention to what you are doing by explaining the need to take notes and involving the interviewee in the process. If necessary, address the issue regarding the confidentiality of working papers.

• Offer the interviewee the opportunity to review the notes taken

during your interview to put him/her at ease with the process, if necessary.

• Keep your working paper pad within easy reach, such as

attached to a clipboard on a desk or table. Always make sure you have a pen handy—and that it works.



• Take notes about facts that are relevant to the audit. Avoid listing points that do not relate to the audit topics of concern. For example: − Record name, title, and job description of person with

whom you spoke. − Reference the appropriate protocol step(s) addressed in

the interview. − Note relevant interview information. − Highlight key statements/observations.

• Put the interviewee’s words in quotation marks to distinguish

them from your own comments. • Spend time immediately following the interview summarising

the key points obtained from the interview in your working papers

Exercise 2 Effective Interviewing - Difficult Interview Situations


Objective

The purpose of this exercise is to identify different types of interview situations that could potentially occur while conducting an audit and to discuss various strategies useful in resolving difficult interview situations. Instructions

Read the following scenarios and answer these questions for each: • What is happening in this interview? • What would you do in this situation? Scenario A

You are responsible for assessing the environmental training provided to site personnel during an environmental audit of the company Mining Samples. During an interview with the lab training co-ordinator , Jo Lopes, you ask ‘Could you tell me what types of environmental training site employees receive?’ Mr. Lopes responds ‘Sure. All site employees attend an initial eight-hour environmental awareness course during their new employee orientation week. Employees also attend and must successfully pass a first aid/cardiopulmonary resuscitation course.’ You say, ‘Getting back to the environmental awareness course, can you describe the types of information that are discussed in this training?’ Mr. Lopes replies, ‘Sure. We train new site employees on environmental issues such as pollution prevention and air and water contamination. Our first aid courses instruct site employees on how to respond to health and safety emergencies that may occur here at Mining Samples. Let me get a copy of the training material for you.’



Scenario B

During an internal audit of the company Big Motors, you are in charge of evaluating the company’s health and safety management systems. The facility manager suggests you talk with Harry Thomas, the facility health and safety co-ordinator to obtain a detailed understanding Big Motor’s health and safety programmes. After leaving three messages with his secretary, Mr. Thomas returns your call. You explain to Mr. Thomas that you wish to meet with him at a mutually convenient time. He responds, ‘I don’t have any free time to meet in person. Why don’t you ask your questions now?’ You begin, ‘Could you describe in general the different health and safety programmes that are in place at this facility?’ Mr. Thomas states, ‘You can find all that information in the site health and safety manual.’ You ask, ‘ Where could I find a copy of that manual?’ Mr. Thomas replies, ‘I’m not sure. There’s probably one floating around in my office somewhere.’ You then inquire, ‘Well, could you briefly describe your duties to me?’ Mr. Thomas answers, ‘I’m in charge of all the programmes listed in the site health and safety manual.’ Scenario C

As the auditor responsible for assessing the management of hazardous wastes at Gesher Manufacturing, you are in the middle of a scheduled interview with Richard Seaton, the facility maintenance manager. Although slightly preoccupied, Mr. Seaton has been responding amicably to your questions. As you listen to Mr. Seaton explain the major product lines at the facility, you recall the environmental co-ordinator, Jessica Schaeffer, did not mention any site waste recycling activities. You ask Mr. Seaton, ‘Does the facility participate in any types of pollution prevention or recycling activities?’ Mr. Seaton’s demeanor rapidly changes as he responds, ‘No. I don’t believe in those types of new-fangled practices.’


NPC/20365/Mod6_158Audit_Skills_Training_Handbook.doc c 90 Copyright © 2005 by Arthur D. Little Ltd. All rights reserved.

Scenario D

On the last day of the audit of the Orecrush Mill water pollution control programme, you realise you have a specific question about the site’s backflow prevention devices. You decide to call Mr Ahraby, the wastewater treatment operator you spoke with earlier in the week. After calling the treatment plant, you learn that Mr. Ahraby is out sick and that Mr Karami, one of the on-duty water treatment operators, is covering for Mr. Ahraby. You ask to speak with Mr Karami; after he answers the telephone, you explain that you spoke with Mr. Ahraby at length earlier in the week and have a specific question regarding water backflow prevention devices. Mr Karami answers, ‘I’d be happy to answer any of your questions. In Mike’s absence, I’m responsible for the entire wastewater treatment plant. Do you know this plant handles 20,000 gallons of wastewater a day, seven days a week? That’s over seven million gallons annually.’ You interrupt, ‘How interesting. Can you tell me if the backflow prevention devices are routinely inspected?’ Mr Karami replies, ‘Sure. Why just last week we had a full plant inspection. The inspectors even made us clean out our personal lockers. I’ll tell you, that took some doing. I had so much stuff in my locker that I couldn’t even get it open . . . ‘



Scenario E

As part of a four-person audit team assigned to audit D&R Chemical’s facility, you are in the process of interviewing Janet Kowalski, the facility health and safety co-ordinator. In an attempt to obtain information regarding site personnel health and safety training from Ms. Kowalski, you ask, ‘Could you explain how the facility ensures that new employees receive appropriate health and safety training?’ Ms Kowalski answers, ‘How do you think the facility should monitor such training.’ You respond, ‘I’m really trying to understand how this particular facility handles personnel health and safety training. Yesterday, I spoke with site training co-ordinator, John Golder, who suggested I talk with you. Could you please describe your training responsibilities to me?’ Ms Kowalski replies, ‘What did the training co-ordinator tell you that I do?’

Exercise 2 – Potential Answers Effective Interviewing - Difficult Interview Situations


The following are general techniques to use in any difficult interview situation: • Establish and maintain mental and physical control of

yourself. • Vary your questioning pattern to obtain maximum information

from the interviewee • Make sure you are adequately prepared for the interview

beforehand so you portray confidence and can maintain control of the interview

• Don’t be afraid to openly acknowledge the difficulty, take a

break, or ask if you can reschedule for a later time Some additional techniques to apply to specific interview situations include: • If the interviewee becomes hostile and aggressive, remain

calm and do not worsen the problem. Limit your questions to just the facts, and convey to the interviewee that the purpose of the interview is to uncover the truth

• If the interviewee can’t stop talking, do not be afraid to politely

interrupt in order to move on in the interview • If the interviewee goes off course, sum up what the

interviewee has said and either move onto the next question or return to the point where things went wrong in the interview

Exercise 3 Effective Interviewing - Conducting Interviews


Objective

The purpose of this exercise is to practice questioning patterns that elicit the maximum amount of relevant information from the interviewee. We will also want to pay attention to planning the interview and to opening the discussion. Instructions

In this exercise, we will role play an interview between an auditor and a plant representative. If you are No. 1 you will be the Observer for this role play. If you are No. 2, you will be the Auditor. If you are No. 3, you will be the Plant Representative. There are different sets of instructions for your assigned role, which are on the following pages: Health and Safety Specialists No. 1 Observer Page 94 No. 2 Auditor Page 95 - 97 No. 3 Plant Representative Page 98 - 99 Environmental Specialists No. 1 Observer Page 100 No. 2 Auditor Page 101 - 102 No. 3 Plant Representative Page 103 - 104 We will break into groups (Observers, Auditors, and Plant Representatives in separate rooms). You will have 10 minutes to prepare for your role, 15 minutes to conduct the interview, and 5 minutes for feedback.

Exercise 3A Effective Interviewing - Conducting Interviews: ‘Lockout/tagout’


Observer’s Instructions

Your role in this exercise is to observe the interview being conducted by the auditor, to note the techniques being used, to document the information in your working papers, and to provide feedback to the auditor and plant representative on your overall impressions of the interview at its conclusion. The auditor’s assignment is to understand and document how the facility manages its lockout/tagout procedures. The auditor will be interviewing the facility’s safety supervisor. To assist you with noting the techniques used during the interview, an index card with an outline of key interviewing techniques will be provided to you/ You may wish to note the number of leading, yes/no, and open-ended questions asked by the auditor by putting tick marks near these items on the index card each time a question is asked by the auditor. Helpful Hint

In order to minimise the distraction created by your role in this exercise, position yourself off to the side between the other two participants and make your note-taking as unobtrusive as possible. Keep track of the time and signal the auditor to wrap up when the interview period is approaching 15 minutes.



Auditor’s Instructions

Your role in this exercise is to play the part of a safety and industrial hygiene auditor. During your interview with the facility’s safety supervisor, you wish to develop and document your understanding of how the facility manages its lockout/tagout procedures (refer to the protocol steps provided on the following pages). The facility’s lockout/tagout programme is designed to protect maintenance and production personnel and outside contractors from contact with energised equipment during maintenance and overhaul. In its operations, the facility utilises steam to clean process vessels, and electric and pneumatic motors to operate mixers, and has numerous product transfers lined located throughout the plant. Personnel involved in these manufacturing operations can often be exposed to high temperature liquids and electrical hazards due to we floors and working surfaces. You may want to consider the following in preparing for your interview: • Identification of equipment requiring lockout/tagout • Written lockout/tagout procedures • Training of ‘affected’ and ‘authorised’ employees • Issuance and use of locks and tags • Inspections of the lockout/tagout procedures performed • Group lockout/tagout • Contractors Take time now to plan for the interview. Think how you will open, conduct, and close the interview.



Auditor(s)/ Comments

Working Paper Reference

Control of Hazardous Energy (Lockout/Tagout)

13. Examine documentation and interview key personnel to confirm that a written hazardous energy control programme has been developed.

14. Review the written hazardous energy control programme, interview affected personnel, and review records. Confirm that:

a. Site and local management have developed and implemented a written hazardous energy control programme for all operations where the unexpected energising, startup, or release of stored energy could occur and cause injury

b. Local management has provided the necessary material (e.g. locks, chains, tags) to their employees for implementing the hazardous energy control programme

c. The facility has implemented a training programme of lockout/tagout policies and procedures, including locking isolation valves

d. The facility has established a programme to perform and document periodic inspections (at least annually) of the energy control programme

e. There are procedures for briefing contractor personnel on the emergency control procedures

f. Only authorised employees implement energy control measures





15. Examine lockout/tagout devices dedicated to controlling hazardous energy to confirm that the element is included

a. Standardised throughout the facility

b. Able to withstand environmental conditions

c. Substantial enough to prevent accidental removal

d. Traceable to the employee applying the device

16. Interview employees who work in areas where lockout/tagout devices are used to confirm that they are notified prior to the application and removal of a lockout/tagout device

17. Confirm that there are procedures in place to ensure the continuity of lockout at shift change and during group lockout/tagout activities



Plant Representative’s Instructions

Your role in this exercise is to play the part of the facility safety supervisor, who is fairly co-operative, but is concerned about how the audit findings will be presented to management. Although you answer specific questions, you tend not to readily volunteer information. Your facility does have a written lockout/tagout procedures. In this exercise, you are embarrassed by the fact that the facility does not rigorously enforce its ‘one lock/one key’ policy. Combination locks are sometimes used instead of keyed locks, and several employees may know the combination. Also, because of inadequate labelling of sources, there have been a few recent instances in which a supervisor observed that the wrong energy source was locked/tagged out. Although no actual incidents occurred, this indicated a failure to verify isolation of the equipment. You will withhold this information unless asked a question that requires you, in all honesty, to reveal it. Do not lie, but make the auditor probe. Follow these guidelines when conducting the interview: • Assume one of the following difficult interviewee stances for

part of the review: − Take control of the interview − Become hostile and aggressive − Give inadequate answers and/or become unresponsive − Continue talking throughout the entire interview − Go off course from the subject matter

• Insert a ten-second pause into the conversation. • Give key information that should be recorded and one piece of

superficial information that should not be recorded.



If you are asked difficult questions for which not enough information has been provided to you, please feel free to make up information. Perhaps the next thing to do will be to imagine a facility with which you are familiar when creating your responses. Be imaginative and enjoy it!

Exercise 3B Effective Interviewing - Conducting Interviews: Air Pollution Control



Your role in this exercise is to observe the interview being conducted by the auditor, to note the techniques being used, to document the information in your working papers, and to provide feedback to the auditor and plant representative on your overall impressions of the interview at its conclusion. The auditor’s assignment is to understand and document how the facility manages its air pollution control programmes. The auditor will be interviewing the facility’s environmental co-ordinator. To assist you with noting the techniques used during the interview, an index card with an outline of key interviewing techniques will be provided to you. You may wish to note the number of leading, yes/no, and open-ended questions asked by the auditor by putting tick marks near these items on the index card each time a question is asked by the auditor. Helpful Hint

In order to minimise the distraction created by your role in this exercise, position yourself off to the side between the other two participants and make your note-taking as unobtrusive as possible. Keep track of time and signal the auditor to wrap up when the interview period is approaching 15 minutes.




Your role in this exercise is to play the part of an environmental auditor. During your interview with the facility’s environmental co-ordinator, you wish to develop and document your understanding of how the facility manages its air pollution control programmes (refer to the protocol step provided on the following page). The facility has over 20 sources that require air permits. In addition, the facility has programmes for monitoring the ambient air, preparing emissions inventories, and inspecting control equipment. You may want to consider the following in preparing for your interview: • Emission source identification and inventory, registration, and

permitting • Emission control equipment • Monitoring programmes • Maintenance and inspection programmes • Odour control programme • Indoor air pollution issues • Training • Reporting and recordkeeping Take time now to plan for the interview. Think how you will open, conduct, and close the interview.





Understanding Management Systems

1. Obtain and document your understanding as to how the facility manages its air emission control programmes. Considerations may include:

a. Emission source identification, registration, and permitting

b. Emission control equipment

c. Monitoring programmes

d. Maintenance and inspection programmes

e. Training

f. Reporting and recordkeeping etc.




Your role in this exercise is to play the part of the facility environmental co-ordinator. You are fairly skeptical of the audit being conducted and its usefulness to the corporation. You are very proud of your plant and its environmental record. You have been inspected by the regulatory agency several times in the past year and have received no citations from any of these inspections. You can feel free to share your skepticism of the audit and your perspective that the plant has been and is in great shape. Your facility does have 23 air permits, 26 air sources (stacks from furnesses, process vents and drains), and 6 emission control devices (scrubbers and dust collectors). You do monitor ambient air for fugitive emissions but are not required to report the results. Your maintenance department does inspect all control equipment every six months as specified in your permit applications. You are responsible for completing all emission inventories and submitting them to the regulator. In this exercise, you are embarrassed by the fact that the facility has just recently (last month) undergone expansion that has resulted in the installation of two new ovens for which the facility did not obtain the required installation permits and does not have operating permits. The facility has also added a flare without revising the permit. Also, some of your permits are past their expiration dates and timely re-applications were not submitted. You will withhold this information unless asked a question that requires you to tell the interviewer honestly about these oversights. Do not lie, just make the auditor probe to discover this fact.



Follow these guidelines when conducting the interview: • Assume one of the following difficult interviewee stances for

part of the interview: − Take control of the interview − Become hostile − Give inadequate answers and/or become unresponsive − Continue talking throughout the entire interview − Go off course from the subject matter

• Insert a ten-second pause into the conversation • Give key information that should be recorded and one piece of

superficial information that should not be recorded. If you are asked difficult questions for which not enough information has been provided to you, please feel free to make up information. Perhaps the next thing to do will be to imagine a facility with which you are familiar when creating your responses. Be imaginative and enjoy it!

Preparing Working Papers


Preparing Working Papers Definition of Working Papers


Working papers consist of any item that documents information gathered by an auditor (i.e., rough notes, worksheets, company records and policies, etc.). Working papers serve as the basis for the audit findings. Working papers should contain: • A description of the environmental, health, and safety

management systems in place at the facility. • A description of the specific actions taken to address each

step of the protocol (tests conducted, sources(s) of information, evidence accumulated).

• A summary of the auditor’s findings and observations.

Preparing Working Papers


Audit Planning

Step 5:Report Audit Findings

Step 1:Understand Management Systems

Step 2:Assess Strengths &

Weaknesses of Management Systems

Step 3:Gather Audit Evidence

Step 4:Evaluate Audit Results

Audit Follow-Up

WorkingPapers

AuditReport

Document scope in working papers

Record understanding in working papers

Record assessment of soundness of system design

Note explanation & disposition of all findings

Document audit findingsat exit meeting

Document verification testing plan and results

Preparing Working Papers - Purpose of Working Papers


The overall purpose of preparing working papers is to aid the auditor in providing reasonable assurance that an adequate audit, consistent with programme goals and objectives, was conducted. Working papers clearly document the information gathered by each auditor during the audit, and the information included in these documents should substantiate both compliance and noncompliance areas. Audit working papers, therefore, provide the principal evidential support for the audit report. Audit working papers: • Provide an organised method for ensuring that all audit steps

have been addressed in a manner consistent with the objectives and established procedures of the audit programme.

• Supplement the protocols by providing audit planning details

such as the time budgeted to individual audit tasks and the auditor’s evaluation of the management systems that may have influenced the conduct of the audit.

• Provide a record of tests conducted and evidence

accumulated. • Provide data that support the audit report and that may be

useful in subsequent action-planning and follow-up activities. • Provide information to assist in answering questions that may

arise during subsequent action planning and follow-up. • Provide a basis for quality assurance and aid in the planning,

performance, and review of future audits.

Preparing Working Papers - Retention of Working Papers


Typically, companies will retain working papers according to policies such as the following: • Until the final report and/or corrective actions are completed. • Until the next audit of that facility. • For a particular retention time based on corporate policy (e.g.,

five years).

Preparing Working Papers - Working Paper Standards


The minimum standards for working papers will vary based on the company’s audit programme objectives and working paper retention policy. The principles presented below reflect the types of working paper standards that have been adopted by various companies.

Examples of Working Paper Standards

Methods/actions to complete each protocol step or rationale for not completing the step (e.g., not applicable) are documented.

Potential ambiguities or misleading comments have been clarified.

Facility compliance and noncompliance with applicable requirements are documented.

All exceptions (deficiencies), observations, and local attention items (items not for the formal report) are identified. The rationale for local attention items is documented.

If a “sampling” approach is used, the types and sizes of the “samples” are identified.

Key requirements to be reviewed are identified.

The auditor’s assessment of the management systems is documented.

The sources by which the auditor gains information are identified.

The auditor’s daily goals and the results achieved are documented.

Protocols and working papers are cross-referenced.

The nature and scope of the audit are identified.

Any changes to the scope are documented.

All exhibits are listed on an exhibit list and referenced in the working papers, as appropriate.

All pages are numbered, dated, and initialed. Any changes made by the auditor to the working papers are initialed.

Preparing Working Papers - Confidentiality of Working Papers


Working papers should be carefully safeguarded during and after all auditing activities. Upon completion of the working papers review at the end of the audit, all audit working papers should be stored in a central file under the control of the audit programme manager, until they are destroyed in accordance with an established records retention policy. (A specific retention period should be established at the corporate level for the audit working papers that is consistent with audit objectives and supported by the legal department.) Since working papers represent documentation of the scope and conduct of the audit, they may be needed to support findings in the audit report should questions arise at a later date. Excerpts may be copied upon approval of the audit programme manager; however, working papers should not be copied without prior approval from audit management. Because there is no absolute guarantee that an audit report will not be discovered by an outside party at some future time, careful preparation of working papers offers the best protection against potential future liabilities.

Preparing Working Papers - Techniques for Recording Information


A number of basic techniques to be used in developing thorough and well-organised working papers should be kept in mind. • Write while conducting the audit. Notations that serve as

reminders of key points are helpful in gaining a complete understanding of facility systems or activities. An auditor should avoid relying on his/her memory and putting off documenting items until he/she “has more time.”

• Start each new topic on a new page. Many times an auditor

will obtain additional information even after he/she feels the particular topic has been completely documented. Any additional information or notes to clarify particular items can easily be inserted in existing text, and dated if the information was obtained on a different day from when the page was prepared, if each topic is entered on a new page. Also, cross-references can be added to indicate where additional information on the topic can be found.

• Clearly label each working paper page. Initial, date, and

sequentially number each page. Labelling each page with the protocol step makes it easier for both the auditor and the audit team leader to review the work performed and helps locate specific topics in the working papers. A single notation identifying the relevant protocol step is generally sufficient (e.g., Protocol Step 8b: Off-Site Shipment of Wastes).

• Keep entries factual. Each statement should be based on

sound evidence, with unconfirmed data or information qualified, and speculation and generalities avoided. For example: − Do not say, “It appears that...”; rather, state the facts that

create the appearance. − Avoid extreme language (e.g., “terrible,” “dangerous,”

“incompetent”). − Distinguish clearly between information obtained by word-

of-mouth and information observed, verified, or concluded.



− Document the source of all information.

• Keep entries legible. Although no one’s penmanship is perfect, working papers should be written in a legible hand. Avoid crowding, leave plenty of space, and write only on one side of the page. This practice will aid in the audit team leader’s review of the working papers and will help the audit programme manager confirm that audit programme goals and objectives have been achieved. At a minimum, each auditor should be able to read his/her own notes. In the event of a mixed language team, the team leader will decide what the language to use in the working papers will be.

• Write clearly in an understandable style. An auditor should

strive to write clearly, so that a person not involved in the audit can understand the steps taken and can reach the same conclusions. Avoid uncommon abbreviations.

• Include photocopies of selected documents. An auditor

should sequentially number and reference selected facility documents as “exhibits” in his/her working papers (e.g., Exhibit A1). If any notations on any exhibit are made, they should be documented in the auditor’s working papers. For example: “Exhibit G1 is a copy of the facility’s air pollution control permit. Page 3 of the permit contains notes that I used to confirm all emission sources.”

• Maintain an exhibit list. To keep track of the exhibits, an

auditor should develop and update an exhibit list as each new exhibit is identified and numbered.



• Highlight “to do” items and findings. Many auditors find it convenient to keep a running list of “to do” items (items that call for further investigation or additional information) during the audit. This can be done by either writing them on a separate page or in some way identifying them where they are noted. Any “to do” items should be indicated as complete and then cross-referenced when they have been finalised.

• Develop and use standard “tick marks.” To increase

efficiency in developing useful working papers, many auditors develop standard “tick marks,” or legend codes (i.e., a personal type of shorthand) for many of the more common or cumbersome working paper notations.

Tick Mark Examples

Item needing further auditor attention

Item where subsequent attention has been given and noted on page 15 in the working papers

Potential report or exit interview exception/observation

Exception/observation after reporting to team leader and/or facility

Exception confirmed by auditor (as item 3) on exit meeting discussion sheet

Potential concern later determined by auditor not to be an exception (explanation on page 17 of the working papers)

PCC-15

3

PCC-17

Preparing Working Papers – Format


The handwritten information and photocopied facility documentation that comprise an auditor’s working papers are normally prepared on site during the audit. The following pages illustrate examples of working paper entries and documentation, including examples of flowcharting, organisational charts, interview notes, descriptions of actions taken and tests performed, interim summary, pending and completed “to do” items, and an exhibit list.

Preparing Working Papers – Format


1. Applicable protocol step

2. Date

3. Auditor’s initials

4. Page number

5. Source of information

6. Tick marks

7. Referenced exhibit

8. “To Do” items

9. Initialed cross-outs

10. Working paper

references

11. Confidentiality

Preparing Working Papers – Example of Flowcharting


MAC p1 of 675

Auditor’s initials and page #

Preparing Working Papers - Example of Organisational Charts


Preparing Working Papers - Example of Interview Summary Notes


M DG 7 of 42

Preparing Working Papers - Example of Descriptions of Actions Taken and Tests Performed


Preparing Working Papers - Example of an Interim Summary


Preparing Working Papers - Example of “To Do” Items (pending and completed)


Preparing Working Papers - Example of an Exhibit List


MDG - 85

Preparing Working Papers - Review of Working Papers


Working papers should undergo a two-stage review process: (1) on-site review by each individual auditor of their own working papers, and (2) post-audit review by the team leader of each auditor’s working papers. 1. On-Site Review

Throughout the audit, each auditor should review his/her working papers frequently (i.e., at least daily and preferably several times during each audit day) to ensure that all audit tasks have been completed, open items have been resolved, and adequate evidence has been gathered to support the findings. The information included in the working papers should cover all elements of the audit protocol and leave no unanswered questions or open items. Working papers should be complete, free-standing records of the actions taken by the auditor that can be used to verify and document compliance and noncompliance situations. 2. Post-Audit Review

Immediately following each audit, and before the draft audit report is issued, all working papers should be reviewed by the audit team leader. This review provides for a quality assurance check on both the individual auditor and the audit topic(s) covered. The reviewer should document his/her review by signing and dating each set of working papers reviewed. The audit report and the exit meeting discussion sheets should be included in this review to ensure that the findings are reported appropriately and are substantiated by sufficient evidence documented in the working papers.

Preparing Working Papers - Review of Working Papers


Sample Checklist to Review Working Papers

Format

Each working paper page is clearly labeled with the applicable protocol step.

Sources of information are clearly identified.

All exhibits are referenced in the working papers.

Each page is sequentially numbered, initialed, and dated.

Cross-outs are initialed; postscripts or afterthoughts are written in a manner that provides appropriate context.

Content

Each protocol step was addressed in accordance with the instructions provided.

Any departures from the protocol are described and explained.

A description of actions taken to complete each protocol step has been documented.

An understanding of how the facility is managing the items under review has been documented.

The conclusions reached as a result of testing have been documented.

All audit findings have been clearly identified.

All findings in the working papers have been included on the audit exit meeting discussion sheet.

Exercise 4 Preparing Working Papers


In this exercise, we will role play another interview between an auditor and a plant representative. The principal objective of this exercise is to practice documenting relevant information obtained from the interviewee. Of course, you will also want to practice the skills from the last exercise – planning the interview, opening the discussion, varying your question patterns, etc. You will have the same role number (1, 2, or 3) as in the previous interviewing exercise. However, the roles have been switched. If your number is ‘1’, you will be the ‘Observer’ for this role play. If your number is ‘2’, you will be the ‘Plant Representative’. If your number is ‘3’, you will be the ‘Auditor’. There are different sets of instruction for each role; please read only the instruction for your assigned role, as listed on the following pages Health and Safety Specialists No. 1 Observer Page 127 No. 2 Plant Representative Page 128 - 129 No. 3 Auditor Page 130 Environmental Specialists No. 1 Observer Page 131 No. 2 Plant Representative Page 132 – 133 No. 3 Auditor Page 134 We will break into groups (Observers, Auditors, and Plant Representatives in separate rooms). You will have 10 minutes to prepare for your role, 15 minutes to conduct the interview, and 5 minutes for feedback.

Exercise 4A Preparing Working Papers – Safety Inspection Programme



Your role in this exercise is to document the interview being conducted and to critique the auditor’s working papers. The auditor’s assignment is to understand and document the implementation of the safety inspection programme as conducted within Department 901’s Superintendent. During the interview, document the responses of the superintendent in your working papers. At the completion of the interview, obtain the auditor’s working papers and: (1) compare them with your own working paper notes (e.g., Identification of topic and interviewee, content and completeness, legibility, etc.); (2) compare them with working paper principles and techniques; and (3) note strengths and weaknesses of these working papers. Helpful Hint:

To minimise the distraction created by your role in this exercise, position yourself to the side of the other two participants and make your note-taking as unobtrusive as possible.




Your role in this exercise is to play the part of the Department 901 Superintendent. You should co-operatively and comprehensively provide information to the auditor. Some information about the equipment safety inspection programme for Department 901 is listed below: • Department 901 has a safety committee (composed of

yourself, two other management representatives) that meets twice per month to discuss safety issues of concern and to tour the department to conduct safety inspections.

• The pressure of production over the last two years have

resulted in certain periods where the committee has failed both to meet and to conduct the inspections

• The result of each inspection are quickly reviewed by the

committee; the two management representatives convey the results to specific work areas for corrective actions.

• No formal corrective action plans are developed but you

believe things are getting addressed. During the interview, you should project a friendly, accommodating demeanor. You are free to develop additional information in responding to the auditor’s questions, and should provide an abundance of information (all of which need not be relevant to the aforementioned topic). The purpose of your role in this exercise is to force the auditor and the observer to think before they write: rather than acting as stenographers who record every word, the auditor and the observer should ideally record only that information which is important and pertinent to the aforementioned topic.



Be imaginative in developing your description of the facility’s safety inspection programme and your responses to the auditor’s questions.




Your role in this exercise is to play the part of a safety and industrial hygiene auditor. During your interview with the Department 901 Superintendent, you wish to develop and document your understanding of how the department manages its safety inspections. You may want to consider the following elements in preparing for your interview: • Responsibility for conducting inspections • Frequency of inspections • Documentation/Reporting of results • Corrective actions/follow-up • Training

Exercise 4B Preparing Working Papers – Spill Response



Your role in this exercise is to document the interview being conducted and to critique the auditor’s working papers. The auditor’s assignment is to understand and document how the facility manages its spill response equipment, distribution, and training programme. The auditor will be interviewing the facility’s emergency spill response team leader. During the interview, document the responses of the facility’s emergency spill response team leader in your working papers. At the completion of the interview, obtain the auditor’s working papers and: (1) compare them with your own working paper notes (e.g., identification of topic and interviewee, content and completeness, legibility, etc.); (2) compare them with working paper principles and techniques; and (3) note strengths and weaknesses of these working papers. Helpful Hint

To minimise the distraction created by your role in this exercise, position yourself to the side of the other two participants and make your note-taking as unobtrusive as possible.




Your role in this exercise is to play the part of the facility’s emergency spill response team leader, who is co-operative and proud of the facility’s emergency spill response team and of its training and performance. Your understanding of the quantities of materials currently stored at the facility is: Tank Farm Acrylonitrile 2 x 15,000 gallons Toluene 1 x 10,000 gallons Fuel Oil 2 x 10,000 gallons Sulfuric Acid 1 x 10,000 gallons Hexane 2 x 10,000 gallons Isobutyl Methacrylate 1 x 5,000 gallons Other Chlorine 6 x 1-ton cylinders manifolded

at wastewater plant 12 x 1-ton full spares Ammonia 1 x 20-ton tank Emulsifiers 55-gallon drums Catalysts 55-gallon drums Other additives 55-gallon drums; 50-pound

bags



Some brief information about the facility is listed below: • Absorbent pads, pillows, along with absorbent granule

supplies are maintained in a central warehouse for distribution upon request.

• The spill team should receive monthly refresher training and

conduct a mock spill event once every six months • The spill team has its own meeting room and storage area for

self-contained breathing apparatus, chemical resistant suits, etc.

• The spill team’s mock spill conducted in August revealed that

some of the newer members of the team were not familiar with procedure and did not know what action to take in the event of a facility spill. As spill team training has been patchy since August, follow-up on this shortcoming has not yet taken place.

During the interview you should project a friendly, accommodating demeanor. You are free to develop additional information in responding to the auditor’s questions, and should provide an abundance of information (all of which need not be relevant to the aforementioned topic). The purpose of your role in this exercise is to force the auditor and observer to think before they write; rather than acting as stenographers who record every word, the auditor and the observer should ideally record only that information which is important and pertinent to the aforementioned topic. Be imaginative in developing your description of the facility’s spill response programme and your responses to the auditor’s questions.




Your role in this exercise is to play the part of an environmental auditor. During your interview with the facility’s spill team captain, you wish to develop and document your understanding of the activities conducted at the facility relating to spill response. You may want to consider the following elements in preparing for your interview: • Hazardous materials on-site • Spill equipment available for use • Storage and maintenance of spill equipment • Spill team training • Mock drills and follow-up

Assessing Strengths and Weaknesses


Assessing Strengths and Weaknesses - Purpose of Assessing Strengths and Weaknesses


Once the auditors have developed an understanding of how the facility manages each of the environmental, health, and safety programmes included within the scope of the audit, the next step is to evaluate the soundness of these management systems in the context of the potential environmental, health, and safety impacts. The purpose of this assessment is to determine verification priorities for Step 3 activities. Spending the time to evaluate priorities will enable the auditor to maximise the effectiveness and efficiency of gathering the evidence needed to achieve the objectives of the audit.

Assessing Strengths and Weaknesses - Process for Assessing Strengths and Weaknesses


Determine the range of potential impacts if a particular HSE issue is not

managed appropriately

Evaluate the management systems to determine if they are designed soundly. That is, consider if the systems, coupled with the controls, are appropriate given the potential impacts

Set priorities for verification so as to provide the optimum allocation of available team resources to ensure that issues representing high risk and weak management / control systems receive sufficient attention

Consider Potential Impacts

Evaluate Management Systems

Set Priorities for Verification

Assessing Strengths and Weaknesses - General Approach for Considering Potential Impacts


Potential impacts refer to the range of potential consequences for the facility or company arising from an event or activity. In evaluating potential impacts, the auditor should ask: “If a health, safety and environmental activity at the facility is not managed appropriately, what consequences could ensue?” Potential impacts might include: • Catastrophic events (e.g., explosion) • Loss of life (e.g., confined space entry procedures) • Injury or illness (e.g., levels of airborne pollutants in the

workplace) • Environmental damage (e.g., spills of hazardous materials to

navigable waters or drinking water supplies) • Legal or financial liability (e.g., air permit violations) • Loss of operation and production • Adverse publicity (e.g., release of visible hazardous toxic

substance to air) • Recordkeeping or reporting exceptions • Occasional spills or releases to the environment • Employee/community exposures to hazardous/toxic

substances • Loss of property (e.g., fire) The auditor should evaluate the potential impacts for each protocol topic based upon a high to low spectrum.

Assessing Strengths and Weaknesses - Assessing Health, Safety and Environmental Management Systems


Evaluating the soundness of a facility’s health, safety and environmental management of key programmes is inherently a subjective process. While regulations tend to stipulate explicit performance or technology requirements, explicit criteria/standards as to what constitutes an adequate management system are only beginning to emerge. In some instances, however, a corporation, division, or facility may have developed its own guidelines or policies as to how a particular activity or function is to be managed. In those situations where there are established management criteria—be they regulatory or internal—the auditor can look to the criteria for assistance in assessing the soundness of management systems. In all other instances, the guidance on the following pages provides a framework for assessing the strengths and weaknesses of the health, safety and environmental management systems.

Assessing Strengths and Weaknesses - General Principles for Assessing Management Systems


Given the absence of explicit criteria for assessing management systems, the following guidance may be helpful.

Management System Descriptors

Assessing The facility has a process in place for: 1) identifying and evaluating conditions and/or materials on site that have regulatory implications, and 2) understanding the scope of applicable regulatory requirements. • Assessments are performed to identify potential risks (e.g.,

leaking underground storage tanks or pipes, carcinogens in the workplace).

• Environmental, health, and safety evaluations are conducted for modified products, processes, and operating ventures and signed off by health, safety and environmental staff.

• Programmes are in place to keep abreast of regulatory changes, to interpret the applicability of those changes to facility operations, and to develop procedures to address those changes.

Planning The facility has developed procedures and systems for managing compliance (e.g., plans, procedures, policies). • Appropriate procedures have been established to respond

to unintended events, such as process shutdowns, as well as to notify appropriate groups within the corporation and in the community.

• Usable programme and procedure guidance exists to direct facility activities to achieve health, safety and environmental goals consistently.

• A system has been developed for recordkeeping which provides documentation of health, safety and environmental activities and compliance with governmental requirements and company policy.




Implementing The facility has structures and/or equipment in place to manage or implement compliance (e.g., pH meters, approved vendor lists, methods to address nonconformance situations, etc.). • Controls for equipment and storage areas are maintained in

an operable manner. • Systems are in place to address nonconformance situations

(e.g., permit exceedances, internal alarms). Reviewing The facility has developed a process for periodically reviewing

and monitoring compliance programmes (e.g., self-inspections, audits, supervisory review of data to cross-verify reports). • The functionality of the controls is periodically tested. • Periodic and comprehensive inspection programmes are in

place. • Deficiencies identified during inspections are corrected in a

timely manner. Organisation Clear roles and responsibilities have been established to

manage compliance with applicable regulations. • Roles and responsibilities are clearly understood with

respect to health, safety and environmental functions. • Health, safety and environmental staff have access to

appropriate line management to discuss issues and concerns.

Resources Qualified and sufficient health, safety and environmental staff and/or line personnel are involved in compliance management. • Staffing levels are appropriate to obtain environmental,

health, and safety compliance management goals. • Clear accountability exists for health, safety and

environmental performance. Training Programmes are in place to familiarise staff with the nature

and scope of compliance programmes. • Health, safety and environmental staff have appropriate

education, training, and experience to fulfill assigned duties. • Information and education programmes are sufficient to

enable employees to carry out health, safety and environmental functions.




Documentation/Information Management

The facility has an accessible and orderly recordkeeping system for compliance-related activities (e.g., training, monitoring, governmental correspondence) and mechanisms in place to communicate relevant information between the various levels and functions of the organisation (i.e., memoranda, weekly meetings). • Records are accessible and managed in an orderly fashion. • Sufficient information is reported to management and

outside agencies, as appropriate.



Based on the information gathered during Step 1: Understand Management Systems, the auditor should assess the strengths and weaknesses of the facility’s approach to managing each protocol topic. An example of such an assessment (to verify compliance with a wastewater permit) is illustrated below.

Management System Activity

Strengths Weaknesses

Assessing Hired an outside consultant five years ago to review operations and identify wastewater issues.

Have hard copies (2004) of regulations on site. Corporate HSE provides monthly updates on changes.

Do not have formal procedure to review capital projects for wastewater impacts. Do not keep P&IDs current. Have not reviewed latest stormwater regulations.

Planning Have formal inspection and preventive maintenance (PM) programme for Wastewater Treatment Plant equipment.

Have site-specific wastewater sampling and analysis manual based on procedures in 40 CFR 136.

Have reduced total wastewater volume by 25% in past two years.

Lab chemist prepares draft Discharge Monitoring Report. HSE coordinator reviews report and raw data before sending to plant manager for typing and signature.

Do not have formal inspection and PM programme for laboratory equipment. Do not have written pollution prevention plan. Sampling and analysis manual does not address stormwater sampling. Plant manager’s secretary types final Discharge Monitoring Report for signature, keeps file copies, and does any exceedance reporting. HSE coordinator does not review final documents. Stormwater pollution prevention plan is based on draft general permit.





Implementing Have continuous monitors for wastewater flow, temperature, and pH at outfall. Wastewater Treatment Plant operator is responsible for preventative measures. Inspected and calibrated weekly. Periodically have outside laboratory analyze split samples as quality control check.

Do not do sampling and analysis of process-specific inputs to Wastewater Treatment Plant. Have not identified points for stormwater sampling. Process upsets cause “slug” flow to Wastewater Treatment Plant; no advance notice given. Laboratory wastewater goes to Publicly owned treatment works, not Wastewater Treatment Plant.

Reviewing Reviewed and updated site policies and procedures in June 2000. HSE coordinator spends 50% of time out in plant; sees/hears about changes as they happen.

Do not have formal self-inspection programme.

Organisation Roles and responsibilities are defined in job descriptions for Wastewater Treatment Plant operator, lab chemist, HSE coordinator; well understood.

Operations manager clearly considers wastewater compliance to be the responsibility of others (e.g., Wastewater Treatment Plant operator); end-of-pipe mentality.





Resources Have sufficient number of qualified staff for Wastewater Treatment Plant and lab. Person who does sampling and analysis is trained chemist with >10 years experience. HSE coordinator has degree in environmental engineering.

Training Chemist and HSE coordinator are encouraged to attend outside meetings/courses 2-3 times per year.

Have not trained internal staff on stormwater issues. Have not provided operations personnel awareness training on wastewater issues.

Documentation Excellent laboratory recordkeeping. Issue weekly newsletter to all plant personnel that includes information on compliance with Wastewater Treatment Plant effluent limitations.

Information Management

Have automated system for tracking training requirements. HSE coordinator has automated system to alert him when Discharge Monitoring Report is due.

Do not have formal records retention policy. Do not give notice to HSE coordinator when DMR is actually sent.

Assessing Strengths and Weaknesses - Balancing Management Systems/Controls Strengths vs. Potential Impacts


Your overall judgment reflects, to a large extent, whether the facility has strong enough systems in place to minimise the potential impacts.

Potential Impacts

Systems and

Controls

Systems

and

Controls

Potential

Impacts

Potential ImpactsSystems and Controls

“Heavyweight” Systemto Manage

Low Potential Impacts

“Lightweight” Systemto Manage

High Potential Impacts

(pretty much OK) (may reflect an imprecise understanding of

impacts)

(may be vulnerableto surprises)

Balanced Approach

Often strengths or weaknesses in the management systems are linked to the presence or absence of several of the activities. The key is to identify which are most relevant to achieve a balanced approach. For example, a redundant (“belt plus suspenders”) system may be appropriate for high-impact situations, while a more straightforward (“strong leather belt”) approach may suffice where impacts are not as significant. Poor or deficient management systems can lead to “errors or omissions,” (e.g., error—failure to perform atmospheric testing prior to entry into a confined space; omission—failure to reschedule new employees for training who missed the initial training) and there are some health, safety and environmental situations in which an error and/or omission could have a substantial impact. Strong controls also reduce the risk of either an error or omission.

Assessing Strengths and Weaknesses - Setting Priorities for Verification


Following the completion of the previous tasks, the auditor is now in a position to establish priorities for verification, that is, an approach to gathering data that will provide the auditor with sufficient data to draw conclusions regarding compliance with established standards and the effectiveness of the management systems. Setting priorities for verification should be dependent on the auditor’s evaluation of the strengths and weaknesses of the management systems, combined with an assessment of the potential impacts associated with a particular topic. This approach is based on the premise that strong management systems can (and should) be employed to mitigate otherwise high potential impacts. As a conceptual model, the auditor can “grid” each of the protocol topics covered to assist in ranking the priorities for verification.



In the empty matrix presented below, indicate how you would prioritise your time for the balance of the audit. Put a “1” in the box corresponding to your highest priority, a “2” in the box corresponding to your second highest priority, etc. That is, graph the strength of the management system on the y axis against the potential impacts on the x axis.

Management Systems

Potential Impacts

LowLow High

WeakWeak

Strong

The process of “gridding” assists the auditor in setting the verification priorities. That is, the gridding helps auditors select an approach to data gathering that emphasises an in-depth review of areas where potential impacts are high and management systems are weak.



For example: • If there are significant potential impacts associated with

noncompliance with a particular topic, and the management systems associated with that topic are judged to be weak, the auditor should ensure that sufficient time is allocated to reviewing that topic.

• Conversely, if there are low potential impacts associated with

noncompliance with a particular topic and the management systems associated with that topic are judged to be strong, the auditor need only spend a relatively small amount of time reviewing that topic.

By developing priorities for verification based on an assessment of the strengths and weaknesses of management systems, and potential impacts associated with noncompliance, the auditor helps ensure that the highest priority issues are covered in significant depth during the audit.

Verif

icat

ion

Prio

ritie

s

Level of Effort

High/Weak

High/Strong

Low/Weak

Low/Strong



Auditors should recognise that it is generally not useful, in terms of setting priorities for verification, to conclude that all protocol topics are of equally high (or low) priority. Furthermore, it is generally not true that potential impacts are equally high (or low) and management systems are equally strong (or weak) for all topics in a given protocol. It is important for the auditor to use the Step 2 process to set priorities for Step 3. The assessment of priorities can be discussed as a team to provide a broader perspective of potential impacts and evaluation of the management systems. Following the development of verification priorities, the team should once again confirm that resources have been appropriately allocated to complete the audit.

Exercise 5A Assessing Strengths and Weaknesses


Objective

The purpose of this exercise is to: • Assess the potential strengths and weaknesses of an HSE

programme and the potential impacts if that particular issue is not managed appropriately.

• Prioritise the order in which the protocol elements should be

completed based on your assessment of the management systems and the potential impacts.

Instructions

You have been assigned the Hazardous Waste Management Audit Protocol. During Monday and Tuesday of the week that you are auditing the Woodmount Company, you have interviewed several key people who are involved in hazardous waste management. Attached is a summary of the information that you have compiled as you begin to understand the management systems surrounding hazardous waste management. Based on your understanding of the management systems and on your assessment of the risks involved, rank the following elements of your protocol:

Protocol Element Potential Impacts

(low/high)

Management Systems

(weak/strong) Priority

Hazardous Waste Manifests

Waste Accumulation and Storage

Waste Classification Training



Based on interviews with selected key staff, you have learned the following: • The work force is quite stable and most of the employees on

site have been there for at least five years. • The facility has a hazardous waste generator number which

was obtained about ten years ago. The number is CAG 110 070 001.

• The facility generates hazardous wastes, including used

solvents and oils generated in the process of maintaining and lubricating machinery, support equipment, and machine shop activities.

• Annually, production supervisors are required to review the

waste profiles and initial the file copy to confirm that there have been no process changes that would alter the waste composition. Every five years, the facility hires a recognised consultant to review its waste characterisation programme.

• Hazardous wastes are stored in a locked shed adjacent to the

main building. − The only light in the shed comes from the indirect light

through the window openings near the ceiling. Employees are instructed not to enter the shed after dark. Exterior spot lights are available in the event of a nighttime emergency.

− Floor drains lead to a small sump, which can be pumped out manually if necessary.

− You were told that the facility has an ample supply of sorbent materials, but have not seen it yet.



• The treatment, storage, and disposal facility (TSDF), which is also the transporter, prepares the manifests for the facility when it picks up a waste load. − The TSDF comes whenever the facility calls. − The TSDF leaves the generator copy of the manifest with

the facility. − Six different Woodmount employees have signed

manifests in the last ten months.

• The receptionist keeps the facility’s manifests. − During the previous year, 25 shipments of hazardous

waste were sent off site to the TSDF. − Whenever a signed TSDF copy of a manifest comes in the

mail, the receptionist inserts it in the file with the original generator copy. (The receptionist noted that this usually occurs within a month of the shipment.)

− The current year’s manifests are kept in the receptionist’s files; older records are boxed and sent to a storage area in the manufacturing building.

− The sample manifest you pulled from the file was done on a 1984 form.

• The maintenance supervisor prepares the annual reports by recording the volume of waste per manifest and submits them to the regulatory agency every year by March 1.



• Facility hazardous waste training programmes are conducted once per year. − They last approximately eight hours. − The topics covered include HAZCOM, spill cleanup, and

an overview of hazardous waste management rules, including waste characterisation.

− Training is provided by an outside consultant, who submits a contract every year describing the approach and scope of the services rendered. The contract includes a provision for “back-up” training of staff who are absent from the on-site session.

− An agenda is prepared to reflect the topics to be covered during the day-long session.

− Production, maintenance, and selected clerical staff and supervisors attend the yearly sessions. Attendees are required to sign in for both the morning and afternoon sessions. Copies of the attendance records are kept by the maintenance supervisor.

Exercise 5B Assessing Strengths and Weaknesses


Objective

The purpose of this exercise is to: • Assess the potential strengths and weaknesses of an HSE

programme and the potential impacts if that particular issue is not managed appropriately.

• Prioritise the order in which the protocol elements should be completed based on your assessment of the management systems and the potential impacts.

Introduction

You have been assigned the Process Safety Management (PSM) Audit Protocol. During Monday and Tuesday of the week that you are auditing the Woodmount Company, you have interviewed several key people who are involved in process safety management. Woodmount is subject to the OSHA PSM Standard because it stores and uses more than 10,000 pounds of flammable materials, chlorine, and ammonia on site. The facility has three process areas that are covered by the standard. Two of the process areas use flammable materials only. The third process area uses chlorine and ammonia. Attached is a summary of the information that you have compiled as you begin to understand the management systems surrounding process safety management. Based on your understanding of the management systems and on your assessment of the risks involved, rank the following elements of your protocol:


(low/high) Management Systems


Operating Procedures Mechanical Integrity Training Process Hazard Assessments/HAZOPs



Based on interviews with selected key staff, you have learned the following: • The work force is quite stable and most of the employees on

site have been there for at least five years. • The facility has an objective of keeping all operating

procedures up to date. Every three years, the operating procedures are reviewed and updated as necessary.

• The facility has a standard for how to develop and write

operating procedures. • The supervisor for each department is responsible for

maintaining operating procedures. • The operations manager approves all operating procedures. • The availability of operating procedures is verified during pre-

startup reviews. • Operating procedures are updated by production engineers

and are reviewed by a committee consisting of the safety manager and senior production operators.

• Changes to operating procedures are summarised in memos

which are kept in a separate section of the operating procedures manual.

• The facility does not maintain documentation of maintenance

performed on individual pressure relief devices. • The facility has not completed an inventory of all pressure

relief devices. • Maintenance personnel have not yet received formal training

in plant safety.



• Associated piping to and from raw material and product storage tanks is not included in the facility’s preventive maintenance (PM) programme.

• The facility’s computerised PM system has not been set up to

archive the completion of the PMs, nor does the facility maintain a hard copy of this information.

• The facility has not yet implemented a periodic visual

inspection and/or nondestructive testing programme for its critical equipment (e.g., storage tanks, piping, and process vessels).

• The facility maintains a matrix of all job titles and their

applicable training requirements. • The facility has developed training “blocks” in order to

facilitate consistency. A refresher training programme has not yet been established to provide training every three years to operators. Most operators are due to receive refresher training within the next four to six months.

• The facility’s training programme concludes with a testing of

the employee’s understanding of the material presented using a written exam and a practical.

• The facility has completed process hazard analyses (PHAs)

for all three covered processes using the hazard and operability (HAZOP) methodology.



• The HAZOP studies included representatives from the following facility organisations: − Operations − Maintenance (instrumentation and process) − Engineering − HSE department, if deemed necessary by Engineering

• HAZOPs conducted included formal feedback from operators. • Results of the HAZOPs are not consistently communicated to

personnel.

Exercise 5A – Potential Answers Assessing Strengths and Weaknesses



(low/high)

Management Systems


Hazardous Waste Manifests Low Weak 3

Waste Accumulation and Storage High Weak 1

Waste Classification High Strong 2

Training Low Strong 4

Exercise 5B – Potential Answers Assessing Strengths and Weaknesses


(low/high)

Management Systems


Operating Procedures High Strong 4

Mechanical Integrity High Weak 1

Training High Strong 3

Process Hazard Assessments / HAZOPs High Weak 2

Gathering Audit Evidence


Gathering Audit Evidence - Purpose of Gathering Audit Evidence


The purpose of Step 3 is to gather data to verify that the facility is in compliance with applicable regulations and that HSE management systems are functioning as intended. Auditors gather data during Step 3 by developing and implementing a verification strategy. A verification strategy, in the simplest sense, is an expansion of the Step 2 assessment in that it details: • The specific focus or goal of the data gathering; • The types of data to be gathered and why; and • The range of tests to be performed to confirm the validity of

facility information. This section of the manual explains the process by which (and the factors to consider) to develop a verification strategy that complements the priorities established in Step 2.

Gathering Audit Evidence – Basic Tasks


There are several basic tasks the auditor should follow in formulating a sound verification strategy.

Task Description

1. Evaluate what needs to be done “What do I need to look at and how much time do I have to do it?”

2. Determine the depth and rigor of review

“How deep do I need to dig? How many kinds of evidence do I need to gather?”

3. Select the types of evidence needed and the methods to gather them

“What types of data will I gather and how can I collect them?”

4. Compare practices against requirements

“How does the facility rate against the applicable requirements?”

5. Document results “What information do I need to record to help me remember what I have learned and substantiate my conclusions?”

Each of these tasks is described on the following pages.



1. Evaluate What Needs to Be Done

To initiate the formulation of a sound verification strategy, the auditor should first develop a good appreciation for what he/she needs to review and how much time there is to do it all. As such, the auditor should: • Determine the specific objectives of the protocol steps for

review and then prioritise them. • Review the audit resources. Determine the Specific Objectives of the Protocol Steps for Review and Then Prioritise Them As the auditor develops a verification strategy, he/she may still need to determine the specific objectives of the protocol steps for review and then prioritise them. This task builds on what the auditor has identified as priorities based on the assessment made in Step 2. The task of understanding and appreciating the objectives of the protocol steps can be more subtle than merely identifying a topic. For instance, suppose that an auditor has identified spill response training as an item that must be verified during the audit. In this situation, there could be two different aspects to verifying the training requirement. On the one hand, the auditor may only want to verify that all applicable employees have had training during the past year. Accordingly, the auditor will likely review training records and attendance sheets. Alternatively, if the auditor wanted to verify that training was adequate or appropriate to the level of the trainees, then he/she may take a different approach toward verification, such as talking with employees who were supposed to be trained and/or evaluating the content of the training manuals and comparing them to regulatory requirements and/or industry practices.



Prior to determining the types of data to gather, the auditor should quickly review the protocol steps to develop a full understanding of what needs to be verified. For example, a protocol step might direct the auditor to verify that a written training programme has been developed or to verify the effectiveness of the training programme. Understanding these types of differences will be important when selecting/identifying the types of data to review. Following this step, the auditor should prioritise the steps that will be completed. This task builds upon the priorities established during Step 2. That is, if an auditor determined in Step 2 that priority should be given to reviewing a facility’s respiratory protection programme, he/she now needs to decide what emphasis should be given to the various elements of the respiratory protection programme—verifying that a written programme has been established, training is conducted, inspections are performed, etc. The auditor should base his/her decisions on those elements that will provide the greatest insights regarding the functioning/implementation of the respiratory protection programme. Review the Audit Resources The auditor should review the audit resources to determine whether the audit strategy established in the pre-audit planning stage is still appropriate or if any modification or reallocation of audit resources is necessary to meet the objectives of the assignment. As a team, auditors should discuss whether: Audit team resources have been appropriately allocated to adequately cover all functional areas. Protocol topics do not overlap in such a manner that on-site activities are duplicated by two or more team members.



2. Determine the Depth and Rigor of Review

In formulating a reliable and defensible verification strategy, the auditor needs to determine how much data to collect and how deep to dig—in other words, the depth and rigor of review. To make this determination, the auditor should rely on the assessment of the potential impacts of the issues or activities being reviewed and the assessment of the soundness of the management systems made in Step 2. For example:

Management Systems

Potential Impacts

LowLow High

WeakWeak

Strong

3 1

4 2

Dig Deepest

Dig Deep

Dig Less Deep

Don’t Dig?

After the auditor has determined how deep to dig, he/she is now in a position to select what types of information to gather and how.



3. Select the Types of Evidence Needed and the Methods to Gather Them

The auditor should now determine what types of evidence are potentially available to be gathered and what methods would need to be utilised to gather them. In developing an approach for collecting evidence, the auditor should take into account what he/she has learned thus far in the audit, especially in Step 1, so as to get right to the business of verification. In this task, the following activities take place: • Consider what was learned in Step 1 • Select the type(s) of audit evidence • Decide whether and how to design a test • Develop sampling strategies Consider What Was Learned in Step 1: Understand Management Systems As auditors identify the types of evidence they will gather and the methods to be employed in gathering them, they should consider what was learned in Step 1: Understand Management Systems, because the nature of the environmental, health, and safety activities and management structures at the facility can influence the verification strategies. For example: • If the auditor is trying to confirm that a facility completely lacks

a hazard communication programme, he/she should not expect to find any physical or documentary evidence because obviously there is no written document describing the non-existent programme. Instead, the auditor will base this conclusion on the testimonial evidence.

• If the spill plan has not been updated in four years, then

perhaps the auditor will need to focus efforts on gathering physical data to determine whether and how current activities compare against the elements of the plan.



• If a facility has different operations and/or different organisational structures at one location, then the auditor will want to ensure that any data-gathering activity is representative of each operation. For instance, if a facility has two different product lines, each of which is managed by a different division, then the auditor may need to go to two different sources to verify that employees received safety training. This factor should be taken into account when evaluating how much time is needed to complete a particular protocol step.

Select the Type(s) of Audit Evidence In general, there are four different types of evidence (also referred to as “audit data” or “audit information”) that can be gathered during an audit, and each can affect the quality and reliability of the audit results. Each type of evidence is associated with a particular method, and each has advantages and limitations. • Physical evidence • Documentary evidence • Testimonial evidence • Circumstantial evidence The auditor can strengthen the weight of the evidence by obtaining several different types and sources of evidence/information.



Physical Evidence

Definition Something that is seen or can be touched.

Method Physical evidence is obtained through observation whereby the auditor collects information through physical examination.

Examples High level alarm, secondary containment, label on a container of chemicals, ethylene oxide monitor, etc.

Advantages Physical evidence is usually one of the most reliable and persuasive types or sources of data. In many situations, the physical presence of an object or operation can, by itself, satisfy a particular requirement of compliance and, therefore, observing that physical object or operation is essential during an audit to verify compliance.

Limitations The mere presence of an object or conduct of an operation does not ensure that it is appropriate for the situation, that it is designed and functioning properly, or that it will continue to function.

Documentary Evidence

Definition Something written down on paper or recorded electronically.

Method Documentary evidence is obtained through the collection and ensuing review of something written or recorded.

Examples Facility HSE policies, standard operating procedures, reports, inspection sheets, etc.

Advantages Documentary evidence allows the auditor to “see” the facility’s practices in a formal sense (i.e., the documentation) via a paper trail. As with physical data, the documentation itself is often a requirement for compliance. Furthermore, documentation of something makes a strong argument that an activity is indeed performed.

Limitations Documentary evidence by itself does not tell the auditor that an activity actually took place. Also, its reliability can be questioned since documents can be generated or altered while preparing for the audit.



Testimonial Evidence

Definition Evidence received from interviews with facility representatives.

Method Testimonial evidence is obtained through inquiry and simply involves asking questions, both formally and informally.

Examples Anything told to the auditor by facility personnel.

Advantages Testimonial evidence gained through inquiry is a very efficient method of gathering data during an audit. It is especially helpful when one is acquiring basic knowledge on HSE systems at a facility, trying to clarify contradictory information, or obtaining explanations to unclear items.

Limitations Testimonial evidence obtained from inquiry must consider factors such as the competency of the questioned individual concerning the topic (e.g., the person’s training); the interest the person providing the response(s) has in the subject discussed; any biases that the individual questioned may have; and the logic and reasonableness of the question (i.e., was the question understood and appreciated).

Circumstantial Evidence

Definition Indirect evidence which conveys an overall impression.

Method Circumstantial evidence is obtained through an auditor’s developing a general impression or intuitive feeling about something at the facility.

Examples The order and neatness of records and files, the attitudes of facility personnel, the apparent relevance of facility staff’s background and experience to their HSE responsibilities.

Advantages Circumstantial evidence can be useful in directing where potential deficiencies may lie within a facility’s HSE management systems.

Limitations Circumstantial evidence is the most unreliable type of evidence and, thus, should never be used to verify compliance.



Decide Whether and How to Design a Test Testing in an audit situation refers to a wide variety of activities1 that can be employed to verify that the facility has implemented its HSE systems, programmes, and procedures and that the implementation has been effective in achieving the intended results. Testing leads to increased confidence that what the auditor was told in interviews or saw on tours is, in fact, effective. The focus of testing is frequently on compliance with specific applicable regulatory and internal requirements. Testing is most commonly done on documentary evidence, such as: • Training records • Monitoring data and reports • Material safety data sheets (MSDSs) • Emission source inventories • Inspection and maintenance logs • Incident investigation reports In the case of documentary evidence, the testing can be done through vouching, recomputation, retracing data, and/or confirmation (see the table on the next page). Depending upon the nature of the specific audit step, any one of these four may be appropriate.

1 Testing in this context does not mean effluent or emissions sampling or chemical analysis.



Types of Testing

Vouching This test would uncover errors in reported data and involves following the paper trail back to the raw data. For example, hazardous waste shipments recorded in the annual report might be tracked back to shipping department records and the hazardous waste manifests. This process verifies that all reported data are supported.

Recomputation This test checks for the accuracy of arithmetic calculations. This would include, for example, recalculating the results of employee monitoring to determine time-weighted averages.

Retracing Data This test would uncover omissions in reported data and involves reviewing the original data records to ensure that all results are appropriately reported. For example, pH strip charts might be reviewed to identify all excursions from permit conditions. The auditor would then verify that no excursions were omitted from the monthly Discharge Monitoring Reports.

Confirmation This test seeks written confirmation of something from independent third parties. This test may be used where an auditor cannot physically observe a condition, such as the operation of an automatic sprinkler system.

An important part of the concept of “testing” is that auditors deliberately, independently, and systematically select the specific pieces of evidence they will look at. Examples of tests are provided below. • A test of the MSDS system might involve choosing a sample

of chemicals from the facility’s master inventory, from purchasing or receiving records, or from chemicals observed during tours, and verifying that current and accurate MSDSs are on site for each chemical in the sample. (In contrast, simply noting whether there are books of MSDSs present in various locations during a tour would not constitute a “test”, but an observation).



• An auditor could start with personnel department or payroll records and develop a sample of employees who should have received particular types of training, then review training records to see whether all of these people had been trained.

Although testing is most commonly applied to documentary evidence, it can apply to testimonial or physical (but not circumstantial) evidence as well. For example, the auditor might test the effectiveness of spill response training by gathering testimonial evidence—asking a selected sample of employees to describe proper spill response procedures. Similarly, a sample of emergency eyewash stations might be selected and inspected by the auditor (gathering physical evidence) as a test to determine if they are in good working order and unobstructed. It is important to note that an auditor does not design a formal, rigorous “test” for every protocol step or topic. Conducting testing is likely to be appropriate for the protocol steps/topics associated with “Dig Deepest” and “Dig Deep” ratings (see p.167) resulting from the Step 2 assessment of strengths and weaknesses. For lower priority protocol steps, the auditor may decide to rely on inquiry and observation, rather than testing, in formulating a conclusion. Develop Sampling Strategies Even after developing a sound verification strategy, auditors can still find themselves with more data to review than time allows. When this happens, the auditor, as part of his/her verification strategy, needs to also develop a sampling strategy. Sampling is the tool the auditor utilises to look at a portion of a whole population of items. Like verification, there are strategies for effective sampling that serve to minimise bias and ensure that what the auditor looks at is representative of actual conditions at the audited facility. Indeed, the sampling strategy itself can affect the validity of the data gathered and, consequently, the validity of the conclusions reached. Sampling strategies will be discussed in detail in the next chapter of the manual.



4. Compare Practices Against Requirements

Following the gathering of data, the auditor needs to compare facility practices against applicable performance requirements to determine whether the facility is in compliance with these requirements, and whether there are enough data to evaluate whether systems are being implemented as designed. If the auditor finds deficiencies in compliance and/or the implementation of management systems, he/she should remember that there still may be time to explore potential underlying causes of selected deficiencies (as discussed in the chapter on evaluating audit results). As the auditor compares practices against requirements, he/she begins to enter into Step 4: Evaluate Audit Results. 5. Document Results

Auditors should document their verification strategy as well as the data gathered. In particular, the auditor should make sure to document the following in his/her working papers:

Information What Auditor Documents

Which protocol steps were reviewed and which were not

The verification strategy (i.e., what topics were examined in depth and why).

Any changes to the verification strategy once initiated

If the auditor’s strategy called for review of certain records, but those records could not be reviewed because they were inaccessible, then this fact should be documented along with whatever alternative verification strategies were employed.

What data were gathered and the source

The testing plans that were utilised and the information actually collected/reviewed by the auditor.

Conclusions Summary(ies) of the results of the verification strategy.



Summary: Developing a Verification Strategy

Task Question Asked How to Do it

Evaluate what needs to be done

“What do I need to look at and how much time do I have to do it?”

• Determine the specific objectives of the protocol steps for review and then prioritise them.

• Review the audit resources.

Determine the depth and rigor of review

“How deep do I need to dig? How many kinds of evidence do I need to gather?”

• Determine where you want to engage in a rigorous review of facility programmes and practices, based on your assessment of the following: − High priorities − Low priorities − Where few or no systems exist

Select the types of evidence needed and the methods to gather them

“What types of data will I gather and how can I collect them?”

• Take into account what you learned in Step 1.

• Determine the types of audit information needed.

• Determine the most appropriate methods for collecting the audit data.

• Determine the areas where you need to design a test.

• Determine the most effective sampling strategy.

Compare practices against requirements

“How does the facility rate against the applicable requirements?”

• Determine if you have enough data to evaluate whether systems are being implemented as designed.

• Explore selected deficiencies for underlying cause(s).

• Begin Step 4. Document results

“What information do I need to record to help me remember what I have learned and substantiate my conclusions?”

• Document what was looked at. • Document what was not looked at

and why. • Summarise conclusions in working

papers.

Exercise 6 Gathering Audit Evidence - Developing Verification Strategies


Objective

The purpose of this exercise is to develop a verification strategy to most effectively conduct the Step 3 portion of the audit (Gather Audit Evidence) and to determine how best to manage the remaining time on site. Instructions

Based on the information provided in one of the following scenarios, outline your strategy for verifying that the facility is in compliance with applicable requirements highlighted in the protocol steps provided in this exercise and that management systems are being implemented and followed at the facility. You should assume that you are in fact at Step 3 in the audit process, that you have already spoken with the key HSE personnel, and that you have a general understanding of management systems.

Exercise 6A Gathering Audit Evidence - Developing Verification Strategies: Contractor’s Safety


Background

It is now Tuesday afternoon of a one-week audit (the close-out meeting is scheduled for 10:00 a.m. Friday morning). After spending all day Monday and Tuesday morning learning how the facility manages safety, you feel that you have a good understanding of the systems in place, and you are now ready to go out and “verify.” What You Understood from Step 1 About the Facility’s System for Managing Contractor Safety

The facility, which is subject to the OSHA process safety management standard, has established a contractor/visitor orientation prior to commencing work. The HSE coordinator, human resources department, and maintenance manager have all been trained to provide the orientation. The orientation covers the facility’s safety rules, emergency/evacuation procedures, and hazard communication. In addition, the facility has various modules (i.e. confined space entry, lockout/tagout, and hazardous waste practices) which are added to the basic orientation depending on the type of work the contractor will be engaged in. Once the contractor has completed the orientation, his/her badge is given a colour-coded stamp which indicates that the person has received orientation and the topics covered (e.g., yellow—basic orientation; green—includes lockout/tagout; red—includes confined space entry; and blue—includes hazardous waste practices). Security personnel and all facility personnel are aware of the facility’s contractor orientation programme and are encouraged to check contractor badges to ensure that they have received orientation.



Protocol Steps Relating to Contractor Safety

Contractor/Visitor Safety Programmes

1. Determine if the facility has a programme for contractor activities that have the potential for affecting process safety, including maintenance or repair, turnaround, major renovation, or specialty work on or adjacent to a process. If so, review the programme and confirm that the facility:

a. Obtains and evaluates information regarding the contractor’s safety

performance and programmes when selecting a contractor. b. Informs contractors, prior to the initiation of the contractors’ work at

the site, of the known potential fire, explosion, or toxic release hazards related to the contractor’s work and the process and obtains a signed confirmation that the contractor has received and understood the information.

c. Explains to contractors, prior to the initiation of the contractors’ work at

the site, the applicable provisions of the emergency action plan. d. Develops and implements safe work practices to control the entrance,

presence, and exit of contract employees in process areas. e. Issues appropriate permits to work (hot and cold) as necessary. f. Periodically evaluates the performance of contract employees in

fulfilling their obligations. g. Maintains a contract employee injury and illness log related to the

contractor’s work in process areas.



Instructions

1. Evaluate What Needs to Be Done Review the protocol steps highlighted and determine the specific objectives of those steps. 2. Select the Types of Evidence Needed and the Methods to

Gather Them List the types of evidence that could be gathered to address the highlighted protocol steps. In addition, identify the items or systems you will want to test, and how you will accomplish verification testing. In developing your verification strategies, consider the following: • What evidence could you gather to address the protocol steps

highlighted? • Where or from whom could you gather the evidence? • How could you design a test of the facility’s contractor safety

programme with respect to the highlighted protocol steps? • When you have decided what you could do, decide which of

these data-gathering activities you would do to address the highlighted protocol steps.

3. Consider What Was Learned During Step 1 What factors learned in Step 1 drove your verification strategy?



To assist in developing your verification strategy, you should use the worksheet provided below which will take you through the basic steps as described in this section. Physical

Evidence Documentary Evidence


What evidence would you gather?

Where or from whom would you gather the evidence?

How would you design a test of the facility’s performance with respect to the highlighted protocol steps?

Exercise 6B Gathering Audit Evidence - Developing Verification Strategies: Hazardous Waste Manifests


Background

It is now Tuesday afternoon of a one-week audit (the close-out meeting is scheduled for 10:00 a.m. Friday morning). After spending all day Monday and Tuesday morning learning how the facility manages its hazardous waste, you feel that you have a good understanding of the systems in place, and you are now ready to go out and “verify.” What You Understood from Step 1 About the Facility’s System for Managing Hazardous Waste Manifests

• The facility’s is a large quantity generator of special (hazardous) wastes.

• The HSE coordinator and maintenance manager have been designated to fill out the waste manifests.

• Once the accumulation storage log indicates that wastes must be shipped off site because they are within two weeks of reaching an accumulation time of 90 days, the waste transporter is called and a pick-up is scheduled.

• The log must be periodically reviewed since there are no automatic systems to notify facility personnel that a shipment must be scheduled.

• Once the waste transporter picks up the hazardous waste, the facility copy of the waste manifests is retained by the Human Resources records clerk. The clerk is responsible for filing the waste manifests with the receipt copy once the signed receipt copy from the disposal facility is received.



Protocol Steps Relating to Special Waste Manifests

Hazardous Waste Manifests

1. By reviewing a representative sample of Waste Manifests, verify that the facility has a program in place to accurately prepare these shipping documents and track waste loads from the point of generation to final disposition. In particular:

a. Note whether or not signed Waste Manifests are returned to the

producer from the disposal or treatment facility. For any that were not, document facility actions to locate the waste shipment.

b. Compare the waste streams shipped off site to your list developed previously. For those materials not covered by Waste Manifests during the review period, interview staff in the operating area(s) where the waste is usually produced and determine if the waste was generated during the review period. If so, resolve how this stream was disposed of without being covered by Waste Manifests.

c. Review the Waste Manifests and determine if each box or information entry has been filled out correctly.



Instructions

1. Evaluate What Needs to Be Done Based on the information provided above, review the protocol steps highlighted and determine the specific objectives of those steps. 2. Select the Types of Evidence Needed and the Methods to

Gather Them Identify the types of evidence that could be gathered to complete the highlighted protocol steps. In addition, identify the items or systems you will want to test, and how you will accomplish verification testing. In developing your verification strategies, consider the following: • What evidence could you gather to address the protocol steps

highlighted? • Where or from whom could you gather the evidence? • How would you design a test of the facility’s hazardous waste

consignment note programme with respect to the highlighted protocol steps?

• When you have decided what you could do, decide which of

these data-gathering activities you would do to address the highlighted protocol steps.

3. Consider What Was Learned During Step 1 What factors learned in Step 1 drove your verification strategy?



To assist in developing your verification strategy, use the worksheet provided below which will take you through the basic steps as described in this section. Physical

Evidence Documentary Evidence





Exercise 6A– Potential Answers Gathering Audit Evidence - Developing Verification Strategies


Scenario A: Contractor Safety

Physical Evidence Documentary Evidence



• Badge colours observed on site.

• Contractor activities on site against the facility’s safety rules.

• Contractor training manual.

• Contractor training records.

• Facility contractor safety rules.

• Contractor sign-in log.

• Contract language.

• Contractor understanding of the training materials and facility safety rules.

• Affected facility personnel’s understanding of the contractor safety programme and its implementation.


• Locations where contractor work is in progress.

• HSE coordinator. • Site security. • Contracting /

Purchasing personnel.

• HSE coordinator. • Site security. • Maintenance

manager. • Facility personnel

who manage on-site contractors.

• On-site contractors (if practical).


• Compare a sample of colour badges on contractors to the activity being conducted.

• Review contractor activities in comparison to the facility’s contractor safety rules.

• Compare a sample of contractors on the sign-in log to the training records.

• Obtain a sample of contractor names and the colour badge observed and compare the colour badge to the level of training received through a records review.

• Survey a sample of contractors to test their understanding of the training provided.

• Survey affected facility personnel’s understanding of the contractor safety programme (e.g., badges).

Exercise 6B– Potential Answers Gathering Audit Evidence - Developing Verification Strategies


Scenario B: Hazardous Waste Manifests




• Accumulation start dates on hazardous waste drums.

• Hazardous waste streams generated by the facility.

• Hazardous waste accumulation storage logs.

• Filed Waste Manifests.

• Chemical purchasing or inventory lists.

• Annual hazardous waste reports.

• Purchase orders for waste disposal.

• HSE coordinator’s and maintenance manager’s understanding of Waste Manifests.

• Human resources records clerk’s understanding of consignment note record keeping.

• Affected facility personnel’s understanding of hazardous waste accumulation.


• Hazardous waste accumulation storage area.

• Process areas where potential hazardous waste streams are generated.

• Human resources records clerk.

• Facility personnel responsible for the accumulation area.

• Purchasing personnel.

• HSE coordinator. • Maintenance

manager. • Facility personnel

responsible for the accumulation area.

• Human resources records clerk.

Exercise 6B– Potential Answers Gathering Audit Evidence - Developing Verification Strategies




How would you test the facility’s management systems to address the highlighted protocol steps?

• Compare the drums in the hazardous waste storage area to the information on the accumulation storage logs.

• Check the accumulation start dates on the hazardous waste drums to determine if drums are being stored over 90 days.

• Review facility processes to identify potential hazardous waste streams and compare to the facility’s identified waste streams.

• Check consignment note signed copy from the waste disposal company, where applicable.

• Compare Waste Manifests versus the annual hazardous waste report.

• Compare Waste Manifests versus waste profiles and/or process information to verify correctness of waste classification.

• Compare lists versus Waste Manifests.

• Review purchasing / shipping records and verify that Waste Manifests are on file for a sample of shipments.

• Ask the HSE coordinator / maintenance manager how they obtain the information regarding various wastes generated (i.e., waste characterisation, time generated etc.).

• Ask affected employees how they manage the hazardous waste accumulation area.

• Ask the records clerk how she/he manages the hazardous waste consignment note records (e.g., retention, signed copy from the waste disposal company etc.).

Sampling Strategies


Exercise 7 Sampling Strategies 1


Objective

The purpose of this exercise is to illustrate some key considerations when developing and implementing a sampling strategy. Instructions

Your assignment is to verify that secondary containment has been provided for all bulk storage tanks. The facility’s Spill Plan indicates that there are 30 bulk storage tanks and that all of them have secondary containment. The facility has a self-inspection checklist that lists the 30 tanks. Because time is limited, you have decided to choose a sample of six storage tanks from the facility’s list. The first tank in your sample has no secondary containment. On the way to the second sample location, you discover a tank that is not listed in the SPCC Plan and note that the drain valve in the containment area is locked “open.” The second tank in your sample has a visible crack in the containment. At this point, what do you do? 1. Stop right there. 2. Finish the sample of six. 3. Check the containment at all 30 tanks.

Sampling Strategies – Overview


A fundamental component of any audit situation involves the review of a facility’s HSE management systems and activities to obtain evidence needed to substantiate compliance or noncompliance. Because auditing is basically a check on the overall compliance status of a facility and is conducted over a relatively short period of time, auditors generally do not examine entire populations of records, documents, or employees. Instead, auditors sample populations in order to draw conclusions regarding compliance with performance standards. To help ensure the gathering of appropriate sampling information, the following process is frequently followed by auditors: 1. Determine objective of protocol step 2. Identify population for review 3. Select sampling method 4. Determine sample size 5. Document results Each of these steps is described on the following pages.

Sampling Strategies – Basic Sampling Process


1. Determine Objective of Protocol Step

• Specify what you are trying to confirm. • Consider the nature of the regulatory or internal standard to

accurately identify the boundaries of the population under review.

2. Identify Population for Review

• Estimate size of population through: − Review of selected documents. − Observations made during initial understanding of health,

safety and environmental management systems in place. − Interviews with facility personnel.

• Pay attention to major subsets or key segments of the

population that need to be included in the review. • Define population before starting to sample.



Sampling Methods

Random

Block

Interval

Stratification

Judgmental Probabilistic

3. Select Sampling Method

Judgmental Sampling Judgmental sampling is used to gather examples of deficiencies or problems to support an auditor’s assessment of a weak or improper health, safety and environmental management system. Sampling is directed toward segments of the population where problems are likely to exist. Judgmental sampling cannot be used to draw compliance conclusions about an entire population because it focuses on only a portion or subset of that population. Judgmental sampling can be used as a first step to provide the auditor with an indication of whether to use a probabilistic sampling technique such as random, block, interval, or stratification sampling.



Probabilistic Sampling Random. The objective is to select items by a statistically-based chance. If properly done, each item in the population should have an equal chance of being selected, and there should be no subjective determinations to bias the sample. (Note: For auditing purposes, “random” is not the same as “haphazard.” Haphazard means “characterised by lack of order or planning; aimless; independent of any reasoning process.” An example of haphazard sampling would be closing your eyes and grabbing one file out of a drawer. • Block. The objective is to analyze certain segments of

records or areas of the facility. For example, if files were arranged alphabetically, in numerical order, or chronologically, one or more blocks (e.g., all the E’s, records numbered 51 through 75, or January and June files) could be selected. While the block method is easy to use, it neglects entire segments of the population.

• Interval. The objective of interval sampling is to select

samples at specific intervals (e.g., every nth segment of the population is analyzed) with the first item selected at random. Increased confidence is achieved where several intervals with different random starts are used.

• Stratification. The objective of stratification sampling is to

arrange items by categories (e.g., high versus low effluent volumes; new versus experienced employees; regular versus weekend or off-shift transactions) based on the auditor’s judgment that the probability of finding an exception is different for different segments of the population and/or that there are categories within the population that represent higher inherent risks. Higher risk categories may, thus, receive greater review and testing. Once the population has been stratified, random, block, or interval sampling can be applied to select items within each segment.



4. Determine Sample Size

• There are two ways to determine sample size: − Statistically − Auditor’s judgment

• In most HSE audit situations, it is both appropriate and

adequate to develop sample sizes based upon professional judgment.

• The auditor must be sure that the sample size is large enough

to be representative of the total population.



5. Document Results

• Document rationale for selecting sample. • Document how sample was selected. • Include in working papers:

− Population under review − How and why population was selected − Type of sampling method employed − Reasons sampling method was used − Potential bias in sample − Sample size and reasons for selecting sample size

Example Guideline for Selection of Sample Size

Suggested Minimum Size of Sample Size of Population

A B C

2–10 11–25 26–50

51–100 101–250 251–500

501–1,000 Over 1,000

100% 100% 50% 25% 15% 10% 5% 3%

100% 40% 20% 15% 10% 5% 3% 2%

30% 20% 15% 10% 5% 3% 2% 1-2%

(These percentages do not imply any specific confidence level but are intended as guidelines only.)

A Suggested minimum sample size for a population(s) being reviewed which is

considered to be extremely important in terms of verifying compliance with applicable requirements and/or is of critical concern to the organisation in terms of potential or actual impacts associated with noncompliance.

B Suggested minimum sample size for a population(s) being reviewed that will provide additional information to substantiate compliance or noncompliance and/or is of considerable importance to the organisation in terms of potential or actual impacts associated with noncompliance.

C Suggested minimum sample size for a population(s) being reviewed that will provide ancillary information in terms of verifying overall compliance with a requirement.



Random Number Table

The random number table (on the following page) is intended for use when the auditor has chosen random sampling as the preferred sampling method. (See discussion above on “random sampling.”) To use the table, start at any location on the table; then, moving in a given direction (across, down, or up), identify the numbers in sequence. For example, in randomly selecting 25 of 90 records, designate values of 1 to 90 for each record, then moving along the table in the direction you have chosen, use the first two digits of each number to choose 25 records. Assuming you start in the upper left-hand corner of the random number table, and move down, your first number is 104 (record 10); the second number is 465 (record 46); the third number is 225 (record 22); etc. If you get a duplicate or a number outside the defined range, ignore it. Stop when you have 25 records to review.



Random Number Table 104

465

225

616

061

534

711

568

253

179

015

905

306

898

510

213

333

585

426

301

764

287

465

676

072

491

358

110

145

350

724

569

378

421

906

035

013

733

880

116

378

192

176

602

851

652

432

457

980

813

150

255

972

078

917

186

187

690

125

164

855

337

749

071

127

524

949

232

066

902

263

358

411

325

795

822

919

605

507

011

779

206

638

113

525

215

395

887

561

355

401

687

008

635

643

528

470

707

672

587

015

853

763

163

604

706

440

600

586

114

916

903

109

973

518

602

312

145

769

047

581

069

103

866

972

241

001

064

354

386

565

217

710

207

020

834

762

094

349

851

590

695

643

711

291

508

426

056

727

354

020

309

648

394

813

906

488

184

449

185

781

094

611

710

512

893

041

831

136

591

066

170

076

507

459

990

509

287

590

281

559

517

847

543

851

439

224

825

570

099

333

889

607

056

443

222

456

490

018

948

816

891

151

535

496

150

632

849

055

649

635

939

875

081

774

198

185

987

518

221

215

641

361

949

212

478

986

378

875

680

873

331

524

369

885

907

832

052

239

963

266

496

889

438

144

055

000

269

134

755

916

279

248

713

606

219

210

425

569

289

409

526

856

772

163

553

298

234

461

304

152

182

185

132

003

810

384

079

481

109

696

726

223

700

478

229

322

926

258

059

622

467

610

582

552

995

206

574

146

006

691

534

493

570

141

818

106

323

854

695

482

927

482

139

607

448

715

643

889

616

969

228

024

168

304

649

878

987

029

100

451

326

780

232

002

442

795

826

400

979

699

633

997

261

787

737

146

992

876

977

141

939

320

008

069

443

129

895

368

882

034

889

522

475

921

011

850

947

195

999

112

293

330

741

038

662

946

985

009

542

003

415

174

654

825

340

290

270

670

283

761

566

306

321

341

857

499

241

897

966

625

340

306

749

012

428

963

143

533

332

496

334

676

810

494

652

511

177

256

328

326

270

288

920

946

804

397

271

481

064

257

761

961

596

720

655

041

325

122

141

508

004

264

634

303

292

945

746

139

864

362

526

196

977

546

995

919

636

539

709

694

363

933

977

539

648

019

351

581

541

323

876

073

246

894

657

574

312

047

508

008

915

183

996

157

857

162

170

027

008

438

733

115

354

484

703

563

756

778

964

209

191

633

163

779

729

054

102

530

799

186

176

015

859

709

449

927

357

486

584

055

873

197

366

415

832

675

806

212

654

968

211

709

947

438

558

153

276

148

807

866

914

443

571

513

602

596

286

691

864

995

396

586

375

069

564

079

170

595

568

726

300

263

293

639

059

649

070

912

224

242

587

924

007

175

341

776

444

208

793

306

362

669

114

998

388

128

982

232

704

709

152

347

109

095

183

091

392

700

544

912

995

421

399

110

699

188

181

388

058

521

082

851

744

756

551

521

976

858

741

133

002

609

228

273

132

443

978

929

538

476

278

997

181

104

593

662

638

350

756

793

069

603

073

259

198

580

528

354

864

907

241

930

818

427

988

209

577

623

901

208

841

202

285

407

010

539

337

143

470

380

458

612

021

639

305

112

704

902

106

231

739

724

813

769

137

383

119

997

767

938

570

609

546

276

428

290

627

345

554

223

483

062

166

277

310

945

843

081

315

122

271

299

907

023

540

463

099

091

253

943

153

500

516

415

977

711

954

124

218

395

206

011

804

259

351

224

346

375

887

281

541

719

936

112

082

443

726

154

413



Sampling Strategies

Random Sampling

Block Sampling

Interval Sampling

Stratification Sampling

Select items in entire population by chance (e.g. using a randomNumber table)

Arrange items by certain segmentsor clusters and randomly selectsome clusters as your sample

Select samples at intervals (every nth item starting randomly)

Arrange items by important categories or subsets, then sample within the groups



Objective

The purpose of this exercise is to: • Use each sampling method (random, block, interval, and

stratification) on a set of hazardous waste training records to determine if affected employees have received refresher training within the last year.

• Determine whether there are any advantages or

disadvantages in using one sampling method versus another.

Background

During your audit of the ACCO Chemical Company, you need to verify whether the site is complying with the corporate requirement that all employees who have responsibility for managing or handling hazardous waste have received annual refresher training and that the training is documented. You have learned that there are three categories of employees who are involved with hazardous waste management: • Within each production department, there are designated

operators, who have routine, day-to-day responsibility for managing a variety of wastes that are generated as a result of frequent cleaning of reactors used for batch specialty chemical production. Other production employees, who might on occasion generate hazardous waste, know that they are not to handle it themselves but are to contact one of the designated operators. If the designated operators are not properly trained, the probability of waste mismanagement is high.



• Maintenance workers handle a much smaller number of relatively constant hazardous waste streams on an infrequent basis. The wastes most frequently handled are used oil and spent solvents from cleaning operations. Occasionally, maintenance workers encounter other wastes (when they service plant equipment (pumps, compressors), for example).

• Supervisors have virtually no hands-on responsibility for

waste management. However, because they are ultimately responsible for the actions of the workers they supervise, it is company policy that all supervisors (not just those in production or maintenance) must receive annual training to maintain awareness.

Based on the above job descriptions and the employee roster, you have developed the list shown in Table 1, which indicates that there are 80 employees total who require annual hazardous waste refresher training according to company policy. The plant environmental coordinator has provided you with his list of employees who attended the two most recent refresher training sessions, both held within the past 12 months (see Table 2). Instructions

Depending on the group to which you are assigned, you are to use one of the following methods to create a sample of approximately 25 percent of the 80-employee total (corresponding to the suggested minimum size of sample in column A using the table on page 9-10): • Random • Block • Interval • Stratification



For your sample, determine how many of the employees in each job category have received the required training according to the documented records (Tables 1 and 2 attached). After you have completed your sampling and analysis, we will compare the results of the various methods.



Table 1: Employee List Developed by Auditor

No Job Category Employee Name

Sampling Method Chosen

Trained in Past Year

(Y/N)

R B I S

1 Maintenance Domingo, P.

2 Maintenance Bell, M.

3 Maintenance Cort, A.

4 Maintenance Greeno, L.

5 Maintenance Herald, M.

6 Maintenance Higgins, H.

7 Maintenance Getchell, M.

8 Maintenance Jones, T.

9 Maintenance Monteiro, L.

10 Maintenance Obbagy, J.

11 Maintenance Plunkett, J.

12 Maintenance Reid, R.

13 Maintenance Rotberg, F.

14 Maintenance Savoie, M.

15 Maintenance Sellers, G.

16 Maintenance Young, R.

17 Operator Allen, M.

18 Operator Arnold, E.

19 Operator Bach, J.S.

20 Operator Bateman, R.

21 Operator Butler, L.

22 Operator Catmur, J.






(Y/N)

R B I S

23 Operator Cavalcanti, C.

24 Operator Coburn, T.

25 Operator Davanzo, L.

26 Operator DiBerto, M.

27 Operator Fitch, T.

28 Operator Fletcher, J.

29 Operator Harris, J.

30 Operator Hasselreis, D.

31 Operator Hryciuk, R.

32 Operator Jones, M.

33 Operator Lennon, J.

34 Operator Lewis, S.

35 Operator Loren, S.

36 Operator McGinnes, M.

37 Operator McLean, R.

38 Operator Moody, C.

39 Operator Murray, E.

40 Operator Neville, N.

41 Operator Nott, M.

23 Operator Nureyev, R.

43 Operator Patel, B.

44 Operator Picasso, P.

45 Operator Ryder, W.

46 Operator Schmidt, H.






(Y/N)

R B I S

47 Operator Smith, F.

48 Operator Smith, J.

49 Operator Smith, N.

50 Operator Stone, O.

51 Operator Tallchief, M.

52 Operator Umscheid, M.

53 Operator Voeller, R.

54 Operator Wescott, W.

55 Operator Windsor, C.

56 Operator Yetskalo, V.

57 Supervisor Berstein, L.

58 Supervisor Boehm, P.

59 Supervisor Canton, C.

60 Supervisor Capogna, S.

61 Supervisor Dolittle, E.

62 Supervisor Ferguson, S.

63 Supervisor Fonteyn, N.

64 Supervisor Gedanke, M.

65 Supervisor Harris, R.

66 Supervisor Hedstrom, G.

67 Supervisor Hill, R.

68 Supervisor Hogwood, C.

69 Supervisor Lopez, D.

70 Supervisor Major, J.






(Y/N)

R B I S

71 Supervisor Moore, T.

72 Supervisor Morrison, J.

73 Supervisor Nutty, P.

74 Supervisor Parigot, M.

75 Supervisor Presley, E.

76 Supervisor Robinson, J.

77 Supervisor Shrimpton, J.

78 Supervisor Starr, R.

79 Supervisor Stricoff, S.

80 Supervisor Thomas, M.



Table 2: Training Records by Employee Name

Attendance at First Annual Refresher Training

Employee Name Job Category

Domingo, P. Maintenance

Arnold, E. Operator

Bach, J.S. Operator

Boehm, P. Supervisor

Butler, L. Operator

Capogna, S. Supervisor

Cavalcanti, C. Operator

Coburn, T. Operator

Cort, A. Maintenance

Ferguson, S. Supervisor

Harris, R. Supervisor

Hogwood, C. Supervisor

Hryciuk, R. Operator

Lennon, J. Operator

McGinnes, M. Operator

McLean, R. Operator

Monteiro, L. Maintenance

Morrison, J. Supervisor

Neville, M. Operator

Nott, M. Operator

Obbagy, J. Maintenance

Parigot, M. Supervisor

Schmidt, H. Operator



Attendance at First Annual Refresher Training


Sellers, G. Maintenance

Shrimpton, J. Supervisor

Smith, J. Operator

Stone, O. Operator

Tallchief, M. Operator

Voeller, R. Operator

Young, R. Maintenance



Attendance at Second Annual Refresher Training


Bell, M. Maintenance

Berstein, L. Supervisor

Canton, C. Supervisor

Catmur, J. Operator

Davanzo, L. Operator

DiBerto, M. Operator

Dolittle, E. Supervisor

Fletcher, J. Operator

Fonteyn, M. Supervisor

Hasselreis, D. Operator

Higgins, H. Maintenance

Hill, R. Supervisor

Lewis, S. Operator

Loren, S. Operator

Moody, C. Operator

Moore, T. Supervisor

Murray, E. Operator

Nutty, P. Supervisor

Nureyev, R. Operator

Plunkett, J. Maintenance

Robinson, J. Supervisor

Savoie, M. Maintenance

Smith, N. Operator

Starr, R. Supervisor

Stricoff, S. Supervisor



Attendance at Second Annual Refresher Training


Thomas, M. Supervisor

Umscheid, M. Operator

Wescott, W. Operator

Yetskalo, V. Operator

Exercise 8 – Potential Answers Sampling Strategies


Determine Objective of Protocol Step

Review facility training records to confirm that employees with hazardous waste management responsibilities have received refresher training within the past year. Determine Total Population and Important Subpopulations

There are 80 employees in three categories who require training: 16 maintenance workers, 40 operators and 24 supervisors. Select Sampling Strategy and Sample Size

1. Random The total population is 80 and you have decided that this is an important issue for the audit. The suggested minimum sample size is therefore 25%. You therefore want to pick 20 random numbers between 1 and 80. One way of generating your sample is to use a random number table. If you start at the lower right-hand corner of the table and read up, the first 20 random numbers (eliminating duplicates) between 1 and 80 are:

13 36 24 6 54 19 51 18 26 41 59 77 43 75 4 15 12 46 11 16

Exercise 8 – Potential Answers (continued) Sampling Strategies


The attached worksheet indicates with an “R” the sample of employees selected by this method. To summarise, the random method gives a reasonably representative sample, with all three job categories included: 7 of 16 maintenance workers 10 of 40 operators 3 of 24 supervisors 2. Block For this type of information, the most logical “block” method to use might be by the first letter of the last name, by generating random numbers between 1 and 26. Using the random number generator on a programmable pocket calculator, you might have selected the numbers : 8, 18, 1, 14, 23 and 4 which correspond, respectively, to last names beginning with H, R, A, N, W and D. A quick count tells you that the first four of these letters will be enough to give a sample of 20. The attached worksheet indicates with a “B” the sample of employees selected by this method. To summarise, the block method also gives a reasonably representative sample, with all three job categories included: • 5 of 16 maintenance workers • 9 of 40 operators • 6 of 24 supervisors



3. Interval To get a 25% sample, you need to select every fourth name on the list. You could choose any number between 1 and 4 as the starting point. The attached worksheet indicates with an “I” the sample of employees selected by this method. To summarise, the interval method also gives a reasonably representative sample, with all three job categories included: • 4 of 16 maintenance workers • 10 of 40 operators • 6 of 24 supervisors 4. Stratified Suppose you decided that annual retraining was very important for the operators, less important for the maintenance workers, and a relatively low priority for the supervisors. The guidelines would then suggest sampling 50% of the operators, 40% of the maintenance workers, and 20% of the supervisors. Applying the interval method within each stratum would require taking every third maintenance worker, every other operator, and every fifth supervisor. The attached worksheet indicates with an “S” the sample of employees selected by this method. To summarise, the stratification method also gives a reasonably representative sample, with all three job categories included: • 6 of 16 maintenance workers • 20 of 40 operators • 5 of 24 supervisors



Analyze Results

1. Random Five of the sample of 20 had not received training, according to the records. Of these, 3 were maintenance workers, 1 was an operator, and 1 was a supervisor. 2. Block Seven of the sample of 20 had not received training, according to the records. Of these, 3 were maintenance workers, 3 were operators, and 1 was a supervisor. 3. Interval Six of the sample of 20 had not received training, according to the records. Of these, 1 was a maintenance worker, 4 were operators, and 1 was a supervisor. 4. Stratification Three of the sample of 6 maintenance workers and 7 of the sample of 20 operators had not received training, according to the records. All of the supervisors in this sample of 5 had received training. (Note: When using this method, it is not correct to combine the results and say “10 out of a sample of 31”.)



Conclusion From This exercise

No two probabilistic sampling methods give exactly the same answer, in quantitative terms. However, qualitatively all four methods give comparable results. The three methods that looked at the population as a whole indicated that between 25% and 35% of the people who should have been trained were not and that persons in all three job categories has missed training. The stratification method indicated that 50% of maintenance workers and 35% of operators had not been trained. This latter method did not pick up the fact that some supervisors also had missed training. However, all four approaches picked up on the fact that a substantial fraction of the people in the two “high risk groups” had not received the training.



Table 1

Employee Roster




(Y/N)

R B I S

1 Maintenance Domingo, P. B S Y

2 Maintenance Bell, M.

3 Maintenance Cort, A. I Y

4 Maintenance Greeno, L. R S N

5 Maintenance Herald, M. B N

6 Maintenance Higgins, H. R B Y

7 Maintenance Getchell, M. I S N

8 Maintenance Jones, T.

9 Maintenance Monteiro, L.

10 Maintenance Obbagy, J. S Y

11 Maintenance Plunkett, J. R I Y

12 Maintenance Reid, R. R B N

13 Maintenance Rotberg, F. R B S N

14 Maintenance Savoie, M.

15 Maintenance Sellers, G. R I Y

16 Maintenance Young, R. R S Y

17 Operator Allen, M. B S N

18 Operator Arnold, E. R B Y

19 Operator Bach, J.S. R I S Y

20 Operator Bateman, R.

21 Operator Butler, L. S Y






(Y/N)

R B I S

22 Operator Catmur, J.

23 Operator Cavalcanti, C. I S Y

24 Operator Coburn, T. R Y

25 Operator Davanzo, L. S Y

26 Operator DiBerto, M. R Y

27 Operator Fitch, T. I S N

28 Operator Fletcher, J.

29 Operator Harris, J. B S N

30 Operator Hasselreis, D. B Y

31 Operator Hryciuk, R. B I S Y

32 Operator Jones, M.

33 Operator Lennon, J. S Y

34 Operator Lewis, S.

35 Operator Loren, S. I S Y

36 Operator McGinnes, M. R Y

37 Operator McLean, R. S Y

38 Operator Moody, C.

39 Operator Murray, E. I S Y

40 Operator Neville, N. B Y

41 Operator Nott, M. R B S Y

23 Operator Nureyev, R. B Y

43 Operator Patel, B. R I S N

44 Operator Picasso, P.

45 Operator Ryder, W. B S N






(Y/N)

R B I S

46 Operator Schmidt, H. R

47 Operator Smith, F. I S N

48 Operator Smith, J.

49 Operator Smith, N. S Y

50 Operator Stone, O.

51 Operator Tallchief, M. R I S Y

52 Operator Umscheid, M.

53 Operator Voeller, R. S Y

54 Operator Wescott, W. R Y

55 Operator Windsor, C. I S N

56 Operator Yetskalo, V.

57 Supervisor Berstein, L. S Y

58 Supervisor Boehm, P.

59 Supervisor Canton, C. R I Y

60 Supervisor Capogna, S.

61 Supervisor Dolittle, E.

62 Supervisor Ferguson, S. S Y

63 Supervisor Fonteyn, N. I Y

64 Supervisor Gedanke, M.

65 Supervisor Harris, R. B Y

66 Supervisor Hedstrom, G. B N

67 Supervisor Hill, R. B I S Y

68 Supervisor Hogwood, C. B Y

69 Supervisor Lopez, D.






(Y/N)

R B I S

70 Supervisor Major, J.

71 Supervisor Moore, T. I Y

72 Supervisor Morrison, J. S Y

73 Supervisor Nutty, P. B Y

74 Supervisor Parigot, M.

75 Supervisor Presley, E. R I N

76 Supervisor Robinson, J. B Y

77 Supervisor Shrimpton, J. R S Y

78 Supervisor Starr, R.

79 Supervisor Stricoff, S. I Y

80 Supervisor Thomas, M.

Evaluating Audit Results




The purpose of Step 4 is to evaluate and summarise the audit results so as to write clear and concise findings that are supported by sufficient audit evidence.

Finding

Exception

There is a problem

Everything is OK

An identified deficiency with respect to a regulatory requirement or a company policy

Good Management Practice

ObservationAn identified weakness with respect to general (industry) standards of good practice in health, safety and environmental management

Management Systems

ObservationAn identified weakness with respect to the processes used by a facility to achieve and maintain conformance with established standards, including programs, policies, equipment, administrative controls etc.

Local Attention Item

An isolated anomaly found in existing programs where regulatory or company standards of performance exist

A conclusion based on documentary,physical, and testimonial evidence, withrespect to health, safety andenvironmental performance

In order to be sufficient to support the audit findings, audit evidence should be: • Relevant • Objective • Persuasive The first two properties relate to the appropriateness of evidence. The last requirement, persuasiveness, refers to its strength.



Relevance

HSE audit evidence should provide a logical basis of support for the findings. As a check on the relevance of the evidence, an auditor should be explicit about the hypothesis he/she is trying to verify. Thus, information developed through a review of the facility’s file of hazardous waste manifests is relevant as verification of the hypothesis that “[a sample of] shipments of hazardous waste are appropriately documented in terms of the EPA’s manifest requirements.” However, the manifest files, by themselves, would not be relevant to verify the hypothesis that “all hazardous waste shipments made by the facility have, in fact, been manifested.” In the latter case, other documents, such as shipping logs, would also be relevant. Objectivity

Audit evidence is objective if it is free from bias. The objective quality of evidence should lead two auditors examining the same evidence to reach the same conclusion. Auditors need to ask themselves (and each other) whether the evidence collected in Step 3 presents an unbiased and, therefore, representative, picture of the true situation. Bias, and a resulting lack of objectivity, could arise from at least three sources: • The auditor’s sampling strategy could introduce bias. For

example, he/she might review all of the incident investigation reports in the Safety Coordinator’s files and conclude that they were being filled out correctly and comprehensively, without realising that there was a completely separate file in the Maintenance Supervisor’s office, which addressed all of the incidents relating to maintenance personnel.



• The auditor could be making some implicit assumptions, based on his/her prior knowledge and experience at other facilities, that are, in fact, not true for the facility being audited. This could introduce an inappropriate subjective element into the evaluation process. For example, an auditor might assume that all maintenance personnel handle hazardous waste and, therefore, require annual refresher training, when this might not be the case.

• Facility personnel could overstate their adherence to internal

procedures if they feel a need to appear more confident and efficient than they really are.

Persuasiveness

Evidence is persuasive when it forces a specific conclusion to be drawn and when another reasonable and knowledgeable person would not challenge the validity of the conclusion nor propose a conceptually different alternative. For example, the evidence that life does not exist on the planet Mercury is very persuasive, but the evidence that there is no extraterrestrial intelligence elsewhere in the universe is less so. In the HSE context, the evidence is persuasive that the facility has a fire extinguisher inspection programme when fire extinguishers are tagged with inspection stickers; the facility has records indicating that inspections are conducted; and the person in charge of the inspections knows when they were last done and when the next inspections are due. The table below indicates some examples of relevant, objective, and persuasive evidence to support particular hypotheses.



Topic Hypothesis Examples of Sufficient Evidence

Data Gathered

• Written hazardous waste training programme has been prepared.

√

• Training records for personnel involved in hazardous waste management are on file.

√

Hazardous waste training programme.

The facility has implemented a hazardous waste training programme.

• Training instructor has the necessary qualifications and expertise to teach the course.

√

• Written respiratory protection programme has been prepared.

√

• Fit testing and medical clearance have been performed.

√

• Periodic inspections of respirators are conducted. √

Respiratory protection programme.

The facility has implemented an effective respiratory protection programme.

• Personnel are trained in the use and maintenance of respirators.

√



Listed below are some basic techniques for the auditor and audit team to consider in evaluating audit results and determining if there are sufficient data to substantiate the findings. As an Individual

1. Confirm the Appropriateness of the Data Gathered Relative to the Audit Scope

As each auditor completes an assigned protocol step—or a group of protocol steps on a particular topic, such as “Spill Response Training”— he/she should take time to review the actions taken to ensure that sufficient data have been gathered. In doing this, he/she should carefully weigh the data against the criteria discussed on the previous pages: • Relevance • Objectivity • Persuasiveness Auditors should also ask themselves whether they have collected an appropriate mix of physical, testimonial, and documentary evidence, using appropriate data-gathering techniques (observation, inquiry, and testing). If a sampling strategy was used, the auditor should ask him/herself one more time whether the sample taken was, in retrospect, the right one. Finally, individual auditors should be sure they have correctly identified the regulatory and internal requirements relevant to the particular protocol topic. 2. Summarise Conclusions Having satisfied him/herself that he/she has gathered sufficient evidence for each assigned protocol topic, the auditor should next summarise his/her conclusions and outline his/her findings.



As a Team

3. Ensure That All Findings Are Substantiated by the Evidence Collected

Having reviewed and evaluated individual audit conclusions, the next step is to meet as a team to review all protocol topics included within the scope of the audit. With the team leader presiding, each of the team members should briefly discuss the actions taken and conclusions reached for each protocol area. As the responsible team member reviews the actions he/she took to address a particular area, other team members should politely and constructively challenge the conclusions. • Critically review the conclusions. It is essential in this step

to critically analyse the rationale for those protocol topics where the team did not identify any findings. For example, if no problems were found with respect to the use of respiratory protection equipment, ask yourselves: “Did we talk to the right people? Are we sure that we accurately identified the affected population? Could there be other plant areas or job tasks where respirators are used (e.g., in the laboratory, unloading rail cars, etc.)?”

• Play “devil’s advocate.” Finally, pause for a moment to ask

yourselves what you could have missed. Given what you have learned about the facility’s programmes, ask yourselves, what is the worst that could happen, and make sure the actions you took to address the protocol area were likely to catch the “worst case,” at least for your “dig deepest” and “dig deep” priorities.



4. Prepare Written Findings The final step in evaluating audit results is to prepare written findings to present to the facility at the exit meeting. The written findings can be presented either in a draft report or simply in a list of the findings. The purpose here is to provide an organised, complete, written summary of the exceptions noted by the team. It is important to present the finding in writing because the true nature of the issue may not be fully appreciated until the facility sees the finding in black and white. As the audit team prepares this document, they should pay careful attention to the following: • Ensure factual accuracy. Because the written findings

provide the basis for the exit meeting with facility management, it is critical that each finding listed be factually accurate. Each team member, in reviewing the wording of the finding, should make sure he/she has the facts to substantiate each exception noted.

• Review with facility HSE coordinator. Prior to the exit

meeting, the audit team should review the written findings with the facility individual who is responsible for day-to-day HSE compliance. He/she will want to know what the team is presenting to “the boss” before it is presented. Also, the coordinator may have some legitimate questions and comments on the findings that the team needs to clarify prior to the exit meeting.



5. Agree Upon Local Attention Items Local attention items should be identified and categorisation agreed upon as a team. Typically, local attention items are those that meet one or more of the following criteria: • Isolated anomalies found in existing programmes where

regulatory or company standards of performance exist (e.g., an occasional signature omission from documents requiring signatures; a single exit sign not lighted).

• Minor items that lack specific criteria (e.g., MSDSs available but not in each of several locations; responsibilities for responding to complaints or incidents not well coordinated).

• Items that are outside the audit scope (e.g., a safety deficiency observed during an environmental audit).

An item should never be for local attention if it is: • An exception to a regulatory or company policy requirement

and is associated with a flaw in the management system. • A repeat of a finding from a previous audit. • An immediate danger to health, safety, or the environment. Example Local Attention Only Items The large alcohol tank in the tank farm does not have a drain plug in place to back up the spring-loaded valve. Two unlabelled 25-liter drums were observed on the concrete pad behind the fire water pump building. (Containers elsewhere at the site were labelled properly.) One wooden ladder being used outside the aerosol gas house was found to be unstable. Facility personnel removed and disposed of it immediately. (The team did not note any other defective ladders.)

Exercise 9 Evaluating Audit Results - Specific “Local” vs. “Report” Decisions


Objective

The purpose of this exercise is to apply the guidelines for classifying local attention items to audit results so as to gain experience in determining whether findings are isolated anomalies or whether the information is sufficient to justify determination as an audit finding. Instructions

On the following pages you will find several sets of findings developed by audit teams. What we have attempted to do is to describe situations that represent varying degrees of “severity” of deficiency with regard to a particular requirement. You are to assume that these situations are mutually exclusive. That is, the auditor observed either the situation described in a. or the situation described in b. or the situation described in c. Also, you should assume that the situation statement describes all of the observed deficiencies with respect to the indicated requirement; there are no other closely related exceptions to the requirement (i.e., if the exercise says you saw on employee not wearing hearing protection, you may infer that all other employees observed were wearing the required protection). You challenge is to work in small groups to decide, within each of these sets of specific findings, where you would draw the line between “local attention” and “for report”. Note that it is quite possible to decide that all three of the versions are in the same category (e.g., all local attention items or all reportable). Note: For purposes of this exercise, do not concentrate too much over whether you would have one or more than one finding based on the information given.



Set 1: Hearing Conservation Requirement: The facility has several areas in which noise levels exceed 90 dBA (8-hour TWA); therefore, hearing protection is required. In addition, it is company policy that signs be posted in these areas and that the entrances to these areas be marked off with pink or green painted lines on the floor. a. A door opens to an air compressor equipment area where

hearing protection is required. The area is posted as “Hearing Protection Required”, but no pink/green marking lines have been marked on the floor.

b. A door opens to an air compressor equipment area where hearing protection is required. The area is posted as “Hearing Protection Required”, but no pink/green marking lines have been marked on the floor. One employee was observed working in the air compressor area with no ear protection.

c. A door opens to an air compressor equipment area where hearing protection is required. The area is posted as “Hearing Protection Required”, but no pink/green marking lines have been marked on the floor. One employee was observed working in the air compressor area with no ear protection. The team also observed one employee (out of ten) in the machine shop and one employee in the pump room (both posted areas) not wearing hearing protection.



Set 2: Ventilation Flow Measurement Requirement: As a matter of good management practice, the facility should conduct and document inspection and maintenance activities on ventilation systems to ensure that filters are changed regularly and that accurate air flow tests are conducted to confirm performance. a. The facility has not measured the air flow for the hood in the

quality control laboratory at any time since installation, when the flow was checked by the vendor.

b. The facility has not measured the air flow for the hood in the quality control laboratory at any time since installation, when the flow was checked by the vendor. There are no flow measurement instruments available on site for testing the performance of ventilation systems used for air contaminant exposure control in the process areas; flows in process areas are checked once yearly by a contractor.

c. The facility has not measured the air flow for the hood in the quality control laboratory at any time since installation, when the flow was checked by the vendor. There are no flow measurement instruments available on site for testing the performance of ventilation systems used for air contaminant exposure control in the process areas; flows in process areas are checked once yearly by a contractor. The site has installed three new ventilation systems since the last contractor inspection; no flow measurements have been made on these systems.



Set 3: Chemical Containment Requirement: The company’s spill control plan states that secondary containment must be provided in all chemical storage areas. a. In the storeroom, there was one five-gallon container (not

empty) overhanging the edge of the spill containment structure.

b. In the storeroom, there was one five-gallon container (not empty) overhanging the edge of the spill containment structure. In this area, there were also ten five-gallon containers set on the floor beside the spill containment structure. The containment structure was full of cans.

c. In the storeroom, there was one five-gallon container (not empty) overhanging the edge of the spill containment structure. In this area, there were also ten five-gallon containers set on the floor beside the spill containment structure. The containment structure was full of cans. In the maintenance area, there were five-gallon drums of solvent with no containment. In the boiler room, there was a 55-gallon drum with no spill containment.



Set 4: Document Control Requirement: As a matter of good management practice, there should be a method of ensuring that critical HSE documents are periodically reviewed and revised as necessary. a. The facility does not have a formal document control

programme to ensure that plans such as the emergency response plan are periodically reviewed, amended, and reissued if necessary.

b. The facility does not have a formal document control programme to ensure that plans such as the emergency response plan are periodically reviewed, amended, and reissued if necessary. The current emergency response plan dates from 1990 and does not include the correct home telephone number for the alternate response coordinator.

c. The facility does not have a formal document control programme to ensure that plans such as the emergency response plan are periodically reviewed, amended, and reissued if necessary. The current emergency response plan dates from 1990 and does not include the correct home telephone number for the alternate response coordinator. It also does not contain specific plans for responding to spills at the aqueous ammonia tank installed in 1993.



Set 5: Management of “Used Oil” Drums Requirement: Company policy states that no more than 55 gallons of used oil is to be stored in working areas. All full drums are to be stored in designated areas, which are to be managed as 90-day accumulation areas. a. One of the six designated “used oil” storage areas showed

evidence of past spills and partially illegible container labels.

b. One of the six designated “used oil” storage areas showed evidence of past spills and partially illegible container labels. Three apparently full drums labeled “used oil” were observed in the maintenance shop and two at the south side fuelling area.

c. One of the six designated “used oil” storage areas showed evidence of past spills and partially illegible container labels. Three apparently full drums labeled “used oil” were observed in the maintenance shop and two at the south side fuelling area. Facility personnel interviewed could not identify individual(s) specifically responsible for “used oil” drum handling.

Exercise 9 – Potential Answers Specific “Local” vs. “Report” Decisions


Set 1: Hearing Conservation

a. local b. report c. report

Set 2: Ventilation Flow Measurement

a. local b. local c. report

Set 3: Chemical Containment

a. local b. report c. report

Set 4: Document Control

a. report b. report c. report

Set 5: Management of “Used Oil” Drums

a. report b. report c. report

You may find that this opinion may change in the light of additional background information. For example the recommended answers for set 2 would be dependent on what the ventilation hood is used for, its toxicity and the frequency of exposure to employees.



Management Systems Information

In addition to the above, there are several other methods the auditor can apply to determine if the data are sufficient to develop a management systems observation. These are: Identify trends among the compliance findings. Perform a cause and effect analysis. This process is generally conducted as a team. 1. Identify Trends Among the Compliance Findings

Once the auditors have developed the complete list of findings, they should, as a team, review the list to develop an integrated, organised summary. In this step, the team should do the following: • Identify common findings. Look for situations where two or

more of the individual findings listed may relate to one basic problem, and may represent a system deficiency.

• Look for patterns or trends. In reviewing the findings, try to

find whether any patterns emerge which suggest that several findings should be combined. Ask yourselves whether there are several findings that, when viewed as a group, may have greater significance.

• Be alert to systemic issues. Ask yourselves whether the

symptoms observed (errors, omissions, etc.) are manifestations of a more fundamental systems weakness.

For example, the audit team may have several findings all related to the same general topic, such as training, and the findings themselves all point to the underlying management systems deficiency that there is no means to identify and track HSE-related training.



Individual findings related to:

Hazardous waste trainingSpill response trainingHazard communication trainingRespirator use trainingConfirmed spaces entry training

Individual findings related to:

Hazardous waste trainingSpill response trainingHazard communication trainingRespirator use trainingConfirmed spaces entry training

The management system deficiency:

No means to identify and track HSE-related training

The management system deficiency:

No means to identify and track HSE-related training

However, management systems observations are not always so easy to recognise during an audit. Thus, the team may want to relate the findings back to the management systems processes. In Step 1, the team gained a lot of insight into how HSE compliance is supposed to be managed at the facility. As the team verified conformance with the facility’s system, they most likely uncovered some gaps and breakdowns. As a means of discerning the management systems deficiencies, the team should try to relate individual findings to one or more of the management processes utilised by the facility to manage its HSE matters. The table on the following page describes how the individual findings can be categorised within the various management processes to assist in developing management system observations.



Compliance Findings Management System Activity

Management Systems Observation

• The facility has not developed an air emissions inventory.

• The facility has not conducted a noise survey.

• The facility has not characterised 12 different waste streams.

• The facility has not conducted workplace monitoring for xyz chemicals.

Assessing The facility has not assessed its HSE issues and risks in several areas, including: • Air emissions inventory • Noise survey • Waste characterisation • Workplace montinoring for

XYZ chemicals

• The facility has not conducted 7 of 12 monthly safety inspections.

• The facility does not document its fire extinguisher inspections.

• Respirators were observed stored out on shop floor.

Reviewing or Organisation/ Resources

There is no comprehensive system for HSE-related inspections. or Responsibility and accountability for the HSE inspection function have not been clearly defined.

2. Perform a Cause and Effect Analysis

Another technique to develop a management systems observation is to identify the underlying causes associated with compliance and good management practice findings. The following examples illustrate this approach.



Finding The facility exceeded its wastewater pH limit on several occasions during the review period. Upon investigation, the team learned that the wastewater treatment system controls had not been maintained regularly and, thus, failed to properly adjust the pH of the effluent. Incident/Effect An exceedance of the pH (level of acid/base) was noted at the wastewater treatment plant. Possible Causes • Failure of pH controls resulting from lack of maintenance • Training • Inspection • Lack of redundant systems • Wrong equipment • System to ensure QA (procedures) • Labor shortage

Finding During the audit, the team observed that the employee’s hand could be pulled into the mobile pumping unit because the unit does not have a guard. Upon investigation, the team learned that the guard had been removed to make access to the equipment easier and a system is not in place to periodically verify that guards are in place where required. Incident/Effect The belt-driven pulley on mobile pumping unit 5123 does not have a guard. Possible Causes • Training • Inspection • Lack of equipment

To make this technique truly useful, the cause-effect model begins at the incident, or effect, and proceeds backward by asking “why” until the appropriate management system conclusion is reached.



Root cause

Cause 3

Cause 2Cause 1 Incident

After the cause (i.e., the main point to communicate to management) is identified, the team is in a position to write a management systems observation.

Writing Audit Findings




As the team prepares the list of audit findings, it is important to ensure that each is written in a manner to clearly and accurately communicate the facts. It is important to keep in mind several principles when wording audit findings: 1. Do Not Overstate the Facts

State the facts as you have discovered them but avoid drawing overly broad conclusions.

Don’t say... If you mean...

The facility does not have a PCB inspection programme.

The facility’s PCB inspection programme does not include PCB-contaminated transformers.

The facility does not have a respiratory protection programme.

The facility’s respiratory protection programme does not include fit testing or routine inspection and maintenance of respirators.

2. Distinguish Between Performance and Documentation

Some regulatory or corporate requirements specify that a particular activity or programme be conducted, but do not specify that the completion of the activity or programme be documented. In other cases, regulatory or corporate requirements specify that the activity or programme be conducted, and that it be documented to verify that it was conducted.

Don’t say... If you mean...

Weekly hazardous waste inspections are not conducted.

Weekly hazardous waste inspections are not documented.

Weekly ladder inspections are not conducted.

Weekly ladder inspections are not documented.



3. Avoid Generalities

Generalities and vague reporting will confuse and mislead the reader. The specific problem should be succinctly communicated. In addition, although a finding may be worded factually, it may not contain enough information to fully communicate the nature and extent of the problem.

Too general More helpful

The facility’s contingency plan is incomplete.

The facility’s contingency plan does not include the following elements: a. Agreement with local authorities. b. Types and location of fire

protection equipment. c. Listing of emergency telephone

numbers.

Employees have not received safety training.

Four of 30 maintenance mechanics have not received lockout/tagout training.

4. Do Not Draw Legal Opinions

Legal judgments, interpretations, and conclusions should be avoided when writing audit findings. Generally speaking, legal conclusions can be characterised by words such as “in violation of…,” “not in compliance with…,” “as required by…,” etc.



Legal conclusion Factual conclusion

Hazardous waste drums were not labeled as required by 40 CFR 262.34.

Drums storing hazardous waste were not labeled with the words, “Hazardous Waste,” or other words indicating their contents. (40 CFR 262.34(c)(1)(ii))

During the review period, the facility was not in compliance with 29 CFR 1910.95.

During the review period, the facility did not conduct annual hearing conservation training. (29 CFR 1910.95)

5. Give Regulatory or Company Policy References

Because the basis for a finding may not always be clear to the report recipient, particularly if the report recipient is an individual who is not involved with environmental, health, and safety issues on a daily basis, regulatory or company policy references should be included.

Poor Improved

Required annual hazardous waste training has not been conducted within the past 18 months.

Annual hazardous waste training has not been conducted within the past 18 months. (40 CFR 265.16 and XYZ Company Policy, HAZWASTE 3.2)

Required annual hearing conservation testing has not been conducted within the past 18 months.

Annual hearing conservation training has not been conducted within the past 18 months. (29 CFR 1910.95 and XYZ Company Policy, HEARCON 4.7)



6. Avoid Extreme Language

Refrain from using such deprecating words as careless, terrible, dangerous, intentional, severe, reckless, incompetent, and the like. These words can be broadly interpreted and are not helpful in communicating the exact nature of the problem.

Poor Improved

The lack of spill containment provisions on the loading dock may lead to a dangerous situation.

Spill containment provisions are not available at the loading dock area where hazardous chemicals are loaded/unloaded adjacent to a drain that discharges to surface waters.

The lack of documented confined space entry procedures for the manufacturing operations may lead to an injury/accident.

The manufacturing operations do not have written confined space entry procedures.

7. Use Familiar Terminology

Not all recipients of the report will be involved in health, safety and environmental activities on a daily basis and, thus, they may not be as familiar with the health, safety and environmental acronyms, abbreviations, and regulatory jargon as the auditors are.

Poor Improved

The facility does not have pollution prevention equipment to prevent exceedances of TSS, BOD, and oil and grease in its discharges to the POTW.

The facility does not have pollution prevention equipment to prevent exceedances of total suspended solids (TSS), biochemical oxygen demand (BOD), and oil and grease in its discharges to the publicly owned treatment works (POTW).

Four of 12 P&IDs reviewed were out of date.

Four of 12 piping and instrumentation diagrams (P&IDs) were out of date.



8. Do Not Focus Criticism on Individuals or Their Mistakes

Do not identify the individual(s) involved in a performance finding or as sources of information. The audit is a review of facility programmes and practices, not of individuals.

Poor Improved

John Doe and Jane Smith were observed . . . .

The team observed maintenance personnel . . . .

9. Avoid Contradictory Messages

Activities or programmes presented in a positive light, when the ultimate message will involve pointing out deficiencies, may confuse the reader and obscure the real message being conveyed.

Poor Improved

Although the facility has a well-written waste analysis plan, it does not include parameters for each hazardous waste analyzed or the frequency of analysis.

The facility’s waste analysis plan does not include the following: a. Parameters for each hazardous

waste analyzed. b. Frequency of analysis.



Poor Improved

Although the facility has a site-specific written hazard communication programme, the hazard communication inventory is not up to date, nor does the facility have a procedure to ensure that employees receive hazard communication training upon initial assignment.

In reviewing the facility’s hazard communication practices, the team noted the following: a. The facility’s hazard

communication inventory is not up to date.

b. The facility does not have a

procedure to ensure that employees receive hazard communication training upon initial assignment.

10. Group Similar Findings

Rather than state several individual findings, look for patterns or trends and group similar findings to more fully convey the message.

Individual Grouped

Quarterly Discharge Report The analysis results for one of 13 samples of the wastewater discharged to the publicly owned treatment works (POTW) was not included in the quarterly discharge report. Reporting of Analytical Results Analytical results for samples taken at internal outfalls were not reported to the POTW.

Reporting of Sample Events to POTW In reviewing quarterly monitoring reports and analytical data, the team noted the following: a. One of 13 samples taken and

analyzed of the wastewater discharged to the publicly owned treatment works (POTW) was not included in the quarterly discharge report.

b. Analytical results of samples taken

at internal outfalls were not reported to the POTW.



Individual Grouped

Labelling Confined Spaces The facility has not posted danger signs or used any other equally effective means to identify the existence and location of and the danger posed by the permit-required confined spaces. Confined Space Permit Programme The confined space permit programme does not include procedures: a. To be taken in the event that a

hazardous atmosphere is detected during entry.

b. For verifying that conditions in the

permit space are acceptable for entry throughout the duration of the authorised entry.

Permit-Required Confined Spaces In reviewing the facility’s confined space entry programme, the team noted the following: a. The facility has not posted danger

signs or used any other equally effective means to identify the existence and location of and the danger posed by the permit-required confined spaces.

b. The confined space permit

programme does not include procedures:

1) To be taken in the event that a

hazardous atmosphere is detected during entry.

2) For verifying that conditions in

the permit space are acceptable for entry throughout the duration of the authorised entry.

11. Write Management Systems Observations

To be effective, management systems observations need to be: • Crisp • To the point (one point at a time) • Clearly framed • Expressed in terms meaningful to managers



What’s Wrong With This Finding?

Clearly Defined Responsibilities A reorganisation of the plant management functions had recently occurred. In conjunction with this reorganisation, a newly created position of Manager, Health, Safety, and Environment Special Projects had been established with the main job activities listed in the personnel job description form. The Safety department organisation has also been expanded. Greater emphasis is being placed on health, safety and environmental issues as evidenced by top plant management now having HSE concerns listed in their formal job description outlines. Plans are under way to expand this emphasis by including these same issues in the job descriptions of first line supervision. In the past, environmental concerns were jointly addressed by the Engineering department and the Maintenance department. The majority of the contacts with governmental agencies were handled by these two departments. Under the recent reorganisation, the Engineering and Maintenance departments have been combined under one manager and most of the previous job activities and responsibilities of both departments have been combined. Although the job duties and responsibilities of the newly created position of Manager, HSE have been defined by management, no accountability document has been prepared to indicate specific responsibility or joint accountability with other departments. In addition, it was not clearly defined who can sign HSE documents and correspondence.

This management systems observation regarding “clearly defined responsibilities”: • Uses 208 words • Includes extraneous information on:

− The reorganisation itself − The Safety department − Greater emphasis on HSE issues − Top management and supervisors job descriptions

• Unnecessarily muddles the basic accountability issue with the

lesser question of who should sign HSE documents and correspondence.



A Sharper Picture Clearly Defined Responsibilities The recent organisation has left undefined the issue of whether the new HSE Manager position shares any joint responsibilities or accountabilities with the now combined Engineering and Maintenance department.

This revised finding is: • Twenty-nine words, or an 86 percent reduction in length. • Crisp and to the point, which is that of joint accountability. • Clearly framed as a management issue, that of clarifying

accountabilities. • Much more easily read, understood, and acted upon.



Example Management Systems Observations

Tracking of Training The facility has not developed a comprehensive system for tracking the movement of employees who change job positions to ensure that regulatory required training is provided in a timely fashion. For example, environmental management staff do not know which employees (some may be security contractors) are responsible for chemical and petroleum product loading and unloading. Promotion of Safety and Health Management Management commitment to the recently reissued health and safety policy does not appear fully evident in that: • A facility inspection programme that includes senior management

participation is not in place. • Promotional activities such as safety/environmental recognition awards,

safety contests, etc., have not been developed. • There are gaps in the attendance of employee, supervisory, and

management personnel in mandatory safety training sessions. • Middle management participation in at least two safety, health, and loss

prevention meetings with their employees per year does not occur. Safety Coordinator Programme In reviewing the facility’s safety coordinator programme, the team noted several issues that impact the effectiveness of this programme. For example: • The roles and responsibilities of the safety coordinators have not been clearly

defined and approved by line management. • The safety coordinators do not regularly attend and participate in safety

coordinator meetings. • Some of the safety coordinators lack the training necessary to perform their

responsibilities in accordance with facility policies and procedures.



Example Management Systems Observations

Management of Health and Safety Issues The management of health and safety issues is not fully integrated into day-to-day operational activities as evidenced by the concerns noted in the areas of contractor compliance, electrical safety, and gaps in the safety coordinator inspection programme. Based on interviews with facility staff, several barriers were noted that hinder the effective integration of health and safety with operations. For example: • The absence of management directives outlining the role of health and safety

staff in the day-to-day administration of compliance programmes throughout the facility.

• The lack of periodic meetings among line management and health and safety staff to discuss and foster integration of health and safety into operations.

In addition, the process for developing and revising safety, health, and loss prevention procedures does not include a formal review by line management to help ensure approval and effective implementation.

12. Consider Using a Template

Consider using a template to prepare a preliminary draft of your findings.

Template for Writing Findings

Template Finding

The Statistical Finding Fifteen of a sample of 30 of the 52 employees who routinely enter confined spaces did not receive training during the last year

[# of deficiencies] of a sample of [# in sample] of [# in universe] of what is wrong

The Have/Do Finding The facility does not have an air pollution permit to operate the three boilers in Powerhouse B on site Facility personnel do not conduct or document inspections of the hazardous waste accumulation area

Who or what / does not / do not have / what they do not have Who / does not do it / does not do (perform) / what they do not do



Template for Writing Findings

Template Finding

The Grouped Finding In reviewing the facility’s practices with respect to flammable/combustible liquid handling, the team noted the following: a. An oxidiser (nitric acid) was

stored in the same flammable liquid storage cabinet as organic solvents

b. Self-closing valves have not been installed on combustible/flammable drums in the dispensing area

c. Devices for bonding dispensing drums to portable metal containers were not in use

[29 CFR 1910.106]

In touring/reviewing / the area toured/document reviewed / the team noted the following:

a. List the exceptions.

The Observed Finding During the facility tour, the team observed an oil sheen near the outfall to the lake

Who or what / observed / what was observed?

The Told Finding The team was told that there have been several spills of hazardous waste to the storm sewers

The team was told that what the team was told

13. Avoid These Words

Adequate Inadequate Some Few Many Not all Sometimes

Occasionally Insufficient Compliance Noncompliance Required Violation

Exercise 10 Writing Audit Findings - Critiquing Audit Findings


Objective

The purpose of this exercise is to review a set of written audit findings and determine whether the findings clearly and appropriately communicate the audit results. Instructions

Based on the principles regarding how to properly word findings, critique the following exit meeting discussion sheets. Also offer suggestions on how to rewrite the findings.



Exit Meeting Discussion Sheet Facility XYZ Facility Discipline HSE

Audit Team Present John Collins, Brenda Fields, Bob Taylor, Jane Smith

Facility Management Present Ralph Gold

Others Present Bill Dunn

Discussion Date MM/DD/YY Prepared by John Collins

# Exception Critique

1 We could not verify that waste manifests were received from TSDFs (Treatment, Storage and Disposal Facility) within 45 days of shipment.

2 Current storage of emergency response equipment may result in increased likelihood of failure.

3 There is minimal on-site compliance with corporate or department contractor safety policy and procedures.

4 Some of the air sources are being operated without proper permits and some are not adequately maintained.

5 The facility’s central MSDS file is very neat and accessible to those employees who should see it. Not all materials used or stored by the facility have MSDSs (Material Safety Data Sheets) in the central file. Those MSDSs reviewed appeared complete and contained the appropriate information.

6 There are no toe boards and missing hand rails.

7 A discrepancy exists among the frequency of safety inspections.



Exit Meeting Discussion Sheet (continued)


8 Ron Kline and Seth McGee were not familiar with the company’s hazard communication programme or could identify where MSDSs were located.

9 The facility’s computer monitoring programme for permit expiration was found to lack a procedure to make sure that all permits were entered into the computer system to begin with.

10 No inspection and maintenance records were available to the audit team and no documented procedures for rail car loading / unloading.

11 There is insufficient personnel to manage all HSE matters given the requirements put forth in the operating manuals which describe the gamut of environmental, health, and safety regulations.

12 The facility often goes through changes in operations which result in additional environmental impacts only to find notifications to permit conditions and variation in employee safety conditions. A management system should be addressed to fix this problem.

Exercise 10 – Potential Answers Writing Audit Findings - Critiquing Audit Findings


Exit Meeting Discussion Sheet Facility XYZ Facility Discipline HSE

Audit Team Present John Collins, Brenda Fields, Bob Taylor, Jane Smith

Facility Management Present Ralph Gold

Others Present Bill Dunn

Discussion Date MM/DD/YY Prepared by John Collins


1 We could not verify that waste manifests were received from TSDFs within 45 days of shipment.

− Implies auditor has not done job properly

− Need to include citation

− What does TSDF stand for?

2 Current storage of emergency response equipment may result in increased likelihood of failure.

− Should not use ambiguous terms (e.g. may, increased)

− So what?

− Which equipment?

3 There is minimal on-site compliance with corporate or department contractor safety policy and procedures.

− Not specific

− Does not describe problem to help site correct it

4 Some of the air sources are being operated without proper permits and some are not adequately maintained.

− Do not use “some” or “proper” or “adequately”

− Be specific; which sources, how many?

5 The facility’s central MSDS file is very neat and accessible to those employees who should see it. Not all materials used or stored by the facility have MSDSs in the central file. Those MSDSs reviewed appeared complete and contained the appropriate information.

− What does MSDS stand for?

− Combines good and bad findings

− Do not use “not all” or “appeared”

Exercise 10 – Potential Answers Writing Audit Findings - Critiquing Audit Findings


Exit Meeting Discussion Sheet (continued)


6 There are no toe boards and missing hand rails.

− So what?

− How many?

− Where?

7 A discrepancy exists among the frequency of safety inspections.

− What is the discrepancy?

− Needs clearer description

8 Ron Kline and Seth McGee were not familiar with the company’s hazard communication programme or could identify where MSDSs were located.

− Avoid using names

9 The facility’s computer monitoring programme for permit expiration was found to lack a procedure to make sure that all permits were entered into the computer system to begin with.

− Bad English

− Irrelevant information

10 No inspection and maintenance records were available to the audit team and no documented procedures for rail car loading / unloading.

− So what?

− What about the procedures for loading / unloading?

11 There is insufficient personnel to manage all HSE matters given the requirements put forth in the operating manuals which describe the gamut of environmental, health, and safety regulations.

− Judgmental not factual

− Do not use “insufficient”

12 The facility often goes through changes in operations which result in additional environmental impacts only to find notifications to permit conditions and variation in employee safety conditions. A management system should be addressed to fix this problem.

− Bad English

− What’s the problem?

− Should the finding include a recommendation?

Post-Audit Activities – Introduction


The objectives and associated responsibilities of the post-audit activities are outlined below.

Objectives Responsibilities • To ensure that the audit results

are clearly communicated to the appropriate levels of management.

Meeting this objective is typically part of the audit team’s responsibility. This is accomplished by means of a formal, written audit report, as described in this section of the manual.

• To ensure that all audit findings are addressed by management through the implementation of a formal corrective action process.

Meeting this objective is typically the responsibility of line management, although the audit team may be asked to make recommendations, review proposed action plans, and/or track the implementation and closure of corrective action.

• To evaluate the effectiveness of the audit and provide suggestions for improving future efforts.

• To share lessons learned during

the audit, especially to similar facilities.

Meeting these last two objectives is frequently among the responsibilities of the audit programme manager and team leader(s).

In this section of the manual, we discuss: • Preparation of audit reports • Protection of audit results • Audit policy statements • Quality assurance in the audit process

Post-Audit Activities – Preparation of Audit Reports


Purpose of Audit Reports

The overall goal of an audit report is to document the audit findings clearly and accurately. Within this overall goal, an audit report has three basic purposes: • To document the scope of the audit and the audit team’s

conclusions regarding the facility’s compliance status. • To provide appropriate levels of management with information

on the results of the audit—information sufficient to meet the needs of the report’s recipients and consistent with the overall objectives of the audit programme.

• To initiate corrective action so that once exceptions to

applicable requirements have been identified, action steps are set in motion to correct the deficiencies found.

A strong linkage exists between the report’s purpose and the overall audit programme’s objective. For example, where the primary objective of the audit programme is to provide assurance to management, the purpose of the audit report is to provide top management with information on the more significant findings. Likewise, when the primary purpose of the audit programme is to provide plant management with information on the environmental, health, and safety status of the facility, the purpose of the audit report is to help facility managers.



Report Recipients

What Are “Appropriate” Levels of Management? Most audit programmes use a hierarchical reporting scheme. • Most, but not all, programmes today require dissemination of

audit reports to high levels of management. • Increasingly, top corporate managers and/or boards of

directors expect to be informed of audit results. • There may, however, be some audit findings (“local attention

items”) that do not require reporting beyond the facility manager level.

An example of a hierarchical reporting scheme is shown on the following page. Example of a Hierarchical Reporting Scheme

Who How What

Facility HSE staff Daily communication All deficiencies noted

Facility manager Exit meeting; draft and final reports

All deficiencies noted

Corporate HSE Affairs; Law Department; division or group management

Draft and final reports All deficiencies noted, except local attention items

Corporate management Periodic status summary Significant matters; overall patterns and trends; general programme status

Board of directors Summary presentation at a board meeting

Overall HSE performance; most significant matters



Report Form and Content

The health, safety and environmental audit report, according to international and industry standards for audit programme performance, must be in writing in order to allow results to be widely communicated, misunderstandings reduced, and follow-up facilitated. Audit reports must be: • Objective • Clear • Concise • Timely The type of information and level of detail provided in an audit report depends upon the objectives of the audit programme, the needs of the report recipients, and the problems identified. An example of a report outline that meets the minimum requirements specified by the Environmental Auditing Roundtable is provided below. I. Introduction

Purpose, date, and scope of the audit Name, location, and description of the audited site Names of the audit team members Criteria utilised in performing the audit Deviations from the planned scope

II. Exceptions and observations discovered during the audit III. General instructions for response and follow-up



Similarly, ISO 14000 (ISO 14010, Section 5.7 and ISO 14011, Section 5.4.2) describes the contents of the audit report as follows:

The audit report should be dated and signed by the lead auditor. The audit report should contain the audit findings or a summary thereof with reference to supporting evidence. Subject to agreement between the lead auditor and the client, the audit report may also include the following: a. The identification of the organisation audited and of the client. b. The agreed objectives, scope, and plan of the audit. c. The agreed criteria, including a list of reference documents against which the

audit was conducted. d. The period covered by the audit and the date(s) the audit was conducted. e. The identification of the auditee’s representatives participating in the audit. f. The identification of the audit team members. g. A statement of the confidential nature of the contents. h. The distribution list for the audit report. i. A summary of the audit process, including any obstacles encountered. j. Audit conclusions, such as:

– EMS conformance to the EMS audit criteria. – Whether the system is properly implemented and maintained. – Whether the internal management review process is able to ensure the

continuing suitability and effectiveness of the EMS. Audit results should be reported in a manner that is clear and easily understood by the recipient of the report. Audit findings need to be described in an appropriate managerial context, which takes into account the recipient’s extent of familiarity with the subject matter. They also need to be free of jargon and unfamiliar terminology. Audit reports should also be factual, unbiased, and free from distortion. Findings should be prepared without prejudice and expressed as pertinent statements of fact, which are supported by sufficient, valid, and documented evidence gathered during the audit. The focus of the audit report should be on the findings developed during the audit and, depending on the scope of the audit programme, recommendations where necessary.



The length of the audit report varies widely, depending on the scope of the audit and the depth of detail needed to meet the recipients’ needs. Potential Components of an Audit Report

A strong relationship exists between the format and content of the audit report and the needs of the report’s recipients. Although audit reports should satisfy the minimum criteria described on the previous pages, there are also a number of potential components to consider. The decision to incorporate one or more of these alternatives into the audit report is really a programme design issue and not within the discretion of an individual auditor or audit team. Nevertheless, auditors need to have a clear understanding of how their company’s audit reports are designed, so that they can contribute effectively to the reporting process. Examples of audit report components are described on the following pages.



Exceptions Only

An exception report is essentially designed to meet the minimum report requirements. The “exceptions and observations” section of such a report is its distinguishing characteristic in that it simply communicates departures from established governmental or internal standards and observations with respect to general (industry) standards of good practice in HSE management. This format does not necessarily provide the report recipients with a means of interpreting the overall significance of the audit findings and may not represent the most desirable option for readers who lack sufficient context to understand and assess the status of the facility’s performance. This alternative is illustrated below.

II. Exceptions and Observations A. Water Pollution Control 1. Sampling Frequency (Regulatory) The facility does not sample National Pollutant Discharge Elimination System (NPDES) parameters such as Biological Oxygen Demand (BOD) and pH on a biweekly schedule. [40 CFR 122.41] 2. Stormwater Pollution Prevention Plan (Regulatory) The facility has not completed a Stormwater Pollution Prevention Plan (SWPPP) and submitted the corresponding certification form to the Bureau of Stormwater Permitting. [NJPDES General Permit NJ0088315] 3. Etc.....



Recommendations

It is becoming more common for an audit programme to assist the facilities by developing suggested recommendations for corrective action to address each of the exceptions and observations. This is particularly helpful when the facility HSE personnel may not be completely familiar with the requirements (especially regulatory requirements) or to facilitate the development of corrective actions within a specified time frame. It is important to ensure that recommendations are written in a way to convey what should be done, without dictating to facility management how it is to be done. Some examples that illustrate this what/how distinction are provided in the table below.

Don’t tell them how Do tell them what

The facility should require that the plant environmental coordinator review all proposed changes to process chemicals.

The facility should establish authorisation procedures to ensure that accountable personnel assess the impacts of process chemical changes on waste stream characteristics.

The facility should hire an industrial hygienist and two additional process safety engineers.

The facility should assess the need for additional industrial hygiene and process safety resources to perform an employee exposure assessment and process hazards analysis in units A, B, and C.



Audit Opinion

An audit opinion provides an overall classification of the facility’s performance in addition to a list of exceptions and observations. An opinion report provides management with a means for interpreting the significance of the audit results and assists them in focusing their resources on areas where improvement is most needed. Arthur D. Little has developed a five-level opinion scheme, as described in the table on the following page. When an audit programme design calls for an audit opinion as part of the report design, it is important to be as explicit as possible about the criteria for assigning each of the alternative opinions. Inconsistency between audit teams in assigning opinions could, for example, result in a facility’s appearing to have “lost ground” in successive audits when the only real difference was two different audit teams with two different interpretations of what “requires improvement” means. There is also an inherent concern of “grade inflation” over time. The criteria for the Arthur D. Little five-level opinion scheme are described in the table. Formulating an audit opinion is frequently the responsibility of the team leader and/or the audit programme manager, with input from individual team members. An audit opinion would not be given in instances where the team’s review was not sufficiently rigorous to substantiate conclusions about the facility’s compliance.



Arthur D. Little’s Scheme

Opinion Criteria

On the basis of its review, the audit team believes that the environmental, health, and safety programmes and practices that were reviewed meet governmental and internal requirements.

The facility is in compliance with all (or virtually all) of the applicable requirements included in the audit scope. Isolated exceptions to a few requirements are noted, but are judged to be occasional, anomalous, and inconsequential in comparison to the level of compliance achieved.

On the basis of its review, the audit team believes that the environmental, health, and safety programmes and practices that were reviewed substantially meet governmental and internal requirements.

Audit results substantiate a high degree of compliance. Only a few requirements are not satisfied. These represent isolated weaknesses in implementation of an otherwise effective compliance programme.

On the basis of its review, the audit team believes that the environmental, health, and safety programmes and practices that were reviewed generally meet governmental and internal requirements, except as noted below.

Several exceptions to applicable requirements are noted. These exceptions are more than anomalies and reflect weaknesses in the design and/or implementation of certain aspects of compliance programmes.

On the basis of its review, the audit team believes that the environmental, health, and safety programmes and practices that were reviewed require improvement to meet governmental and internal requirements.

Several exceptions to applicable requirements are noted. Some of the exceptions reflect the absence of one or two required programmes, significant departures from a few established criteria, or lapses in programme implementation.

On the basis of its review, the audit team believes that the environmental, health, and safety programmes and practices that were reviewed require significant improvement to meet governmental and internal requirements.

Many exceptions to applicable requirements are noted. They included significant departures from various established criteria, the absence of several required programmes, or prolonged inattention to the resolution of previously identified compliance or liability issues.



Strengths

Oral acknowledgment of the facility’s strengths frequently occurs during the team’s daily interactions with facility personnel and in the exit meeting. Increasingly, however, companies are recognising the value of giving credit to areas of real strength in the written report as well by explicitly acknowledging those areas in which the facility is performing particularly well, through design and implementation of an effective management system, which provides some context within which to evaluate the audit exceptions and observations. This acknowledgment is most commonly done in the executive summary section of the report. For example:

In its review of the facility’s programmes and practices, the team noted the following as areas of real strength: • The hazardous waste management programme is clearly understood by

staff throughout the facility and is documented to demonstrate commitment to compliance.

• The bloodborne pathogens programme is well developed, organised, and tracked.

To avoid confusion, it is important that both the auditors and facility personnel have a common understanding of what is meant by a “strength.” A strength need not necessarily be a unique, world class, or extraordinary practice. On the other hand, the mere fact that a facility is in compliance with a regulatory requirement is not an example of a strength.



Executive Summary

The purpose of the executive summary is to provide a one- or two-page section at the beginning of the report that conveys the highlights of the audit to readers. An example of an executive summary is illustrated below.

A health, safety and environmental (HSE) audit was conducted at ABC Company’s XYZ facility on Month/Day/Year. The time period under review was January 1, 199X, through the last day of the audit. The purpose of the audit was to verify compliance with applicable federal, state, and local safety and health laws and regulations. The scope of the audit included air pollution control, water pollution control, solid and hazardous waste management, industrial hygiene, employee safety, and loss prevention. Specific deficiencies in the HSE programmes and practices are described in detail in the body of this report. On the basis of its review, the audit team believes that XYZ facility will need to develop and implement more formal HSE management programmes to achieve the HSE goals recently established by facility management. In the audit team’s opinion, the HSE programmes currently in place are heavily dependent upon the capability and good intention of facility personnel and are being implemented in the absence of clearly defined written procedures and plans. In addition, the level of familiarity with regulatory obligations varied among the staff responsible for HSE compliance activities. The management of XYZ facility in Anytown, Anystate, has initiated a programme to ascertain the status of HSE programmes on site, to develop appropriate corrective actions to remediate problems, and to develop overall systems to ensure compliance with governmental rules and regulations. In addition, the facility has undergone a recent management reorganisation creating a more centralised HSE group. Thus, these activities represent an increased commitment to manage compliance. To develop more formal HSE programmes suitable to the operations of XYZ facility, we recommend the following for consideration: • Developing clearly stated and well-defined descriptions of roles and responsibilities for all

HSE activities, and communicating those to facility staff. • Establishing well-developed plans and written procedures, as appropriate, for undertaking

compliance activities which are communicated and understood by all key HSE staff. • Undertaking a facility programme to periodically review and monitor HSE compliance and

to identify problems. • Developing an active, formal training and awareness programme for key staff in all areas

of HSE requirements. Lastly, we believe the facility should continue in its efforts to identify and assess site contamination and to develop appropriate remedial plans.



Explanation of Requirements

In many instances, the readers of audit reports are not all equally familiar with the regulatory or company policy requirements to which the team has identified exceptions. In these situations, it may be helpful for the audit report to include a brief description of the regulation or company policy that is being cited as the basis for the finding. For ease of reading, it is useful to identify the requirement and the exception separately. For example:

Management of Satellite Accumulation Areas

Description of Requirement

Federal regulations allow for the accumulation of up to 55 gallons of hazardous waste in containers at or near the point of generation and under the control of the operator of the process generating the waste (commonly referred to as a “satellite accumulation area”), without a time limit, permit, or interim status, contingent upon the generator complying with specific container labelling and management conditions. [40 CFR 262.34(c) and 265.171-265.173]

Exception In inspecting a sample of areas where wastes are being accumulated, the team noted the following: • In the Rastex process, solvent-contaminated filters are

routinely changed and disposed of as hazardous waste. The “satellite” accumulation area for filters taken from the RASTEX 679 reactor is located across the manufacturing building at the DACSTAB UVX production area. In the team’s opinion, the DACSTAB UVX area is not “at or near” the RASTEX reactor, nor is it “under the control of the operator” of the RASTEX process.

• In several areas, the team observed containers that were used to collect solvent-contaminated rags. The containers were not marked with the words “Hazardous Waste” or with other words to describe the container contents and were not kept closed during times when waste was neither being added nor removed.

Post-Audit Activities – Audit Policy Statement


One of the most common concerns with audit reports and audit-related documentation is the risk of disclosure in an enforcement or litigation situation. Therefore, many organisations will establish or utilise one or several mechanisms to protect their audit results. Some of the various means that can be used to protect audit results include: • Physical limitations on distribution • Attorney-client privilege • Attorney work product • Audit privilege Physical Limitations on Distribution This protection is not founded in law, but simply requires the audit programme to exercise due care in the release and handling of all information relating to the audit. There are no specific requirements in utilising this protection other than following common sense and whatever systems or procedures are in place in individual organisations for maintaining confidentiality. Attorney-Client Privilege Attorney-client privilege protects the confidential communications between a “person” (who may, in fact, be a company) and his/her attorney. Legal protection under this privilege requires that: • The person asserting the privilege is a client of the attorney to

whom the information is entrusted. • The communication is made to the attorney or someone

working for the attorney (e.g., an audit team). • The attorney is engaged in preparing a factual investigation or

legal opinion for the client.

• The communication is kept confidential and the privilege is not breached voluntarily or inadvertently waived, i.e., information is not shared freely or recklessly.

Post-Audit Activities – Audit Policy Statement


In the context of an HSE audit, the corporation is the client and the audit results (including the report, the working papers, etc.) can be designated as the confidential communications between the client (the facility/ company) and the attorney (via the audit team, who is acting as the agent of the attorney). Attorney Work Product Another potential means of protection of audit results is the work product doctrine. The work product rule protects information/material prepared by an attorney in anticipation of litigation. This privilege, however, is not absolute, and can be vacated by a showing of “undue hardship and substantial need” on the part of the party seeking to discover the privileged information. Privilege under the work product rule is seldom utilised to protect audit results in the course of routine HSE audits. For one thing, most audits are not typically conducted in anticipation of litigation, but rather are used as a management tool to assure or measure compliance. Furthermore, the privilege under the work product rule rests with the attorney, unlike the attorney-client privilege, which rests with the client. Therefore, in the context of HSE audits, this type of privilege would typically require substantial involvement of counsel in the audit process.

Post-Audit Activities - Quality Assurance in the Audit Process

NPC/20365/Mod6_158Audit_Skills_Training_Handbook.doc 277

Copyright © 2005 by Arthur D. Little Ltd. All rights reserved.

Many audit programmes have quality assurance mechanisms such as: • Team leader reviews • Customer/facility feedback • Periodic programme reviews Team Leader Reviews

One important aspect of team leader reviews of auditor performance is working paper review, which was discussed earlier (see checklist in Tab 6). In addition, team leaders frequently evaluate and provide feedback to individual auditors regarding their technical and interpersonal skills as auditors. If a person is less than effective in one area, he/she can receive some training or other assistance in that area prior to or during the next audit. Many companies have formalised this process with questionnaires or forms for the team leader to complete. An example of a feedback form, which is based on the ISO 14012 qualification criteria for auditors, is provided in the table on the following page.




Auditor Name

Facility Audited

Audit Dates

Team Leader/Reviewer

Audit Element Score* Comments

1 2 3 4 5

Technical Knowledge • HSE science and

technology • Facility operations • Regulatory requirements • HSE management systems • Audit procedures and

techniques − Interviewing − Using the protocol − Keeping working papers − Writing findings

Personal Attributes • Clarity in oral

communication • Foreign language capability • Diplomacy, tact, and

listening skill • Independence and

objectivity • Personal organisation and

time management • Ability to reach sound

judgments based on objective evidence

*1 = poor; 3 = average; 5 = exceptional; N/A = not applicable




Customer/Facility Feedback

Upon completion of the field activities, it is quite common for the audit team to solicit feedback from the “auditees.” This may be done informally, for example by means of a debriefing conversation between the team leader and facility management after the closing meeting, or more formally. Companies that solicit such feedback have generally found it to be a valuable mechanism for improving the overall effectiveness and acceptance of the audit programme. Managers of audit programmes have also found, however, that it is important to keep the feedback form simple, so as not to overburden the facility. The table on the following page illustrates the feedback form utilised in one audit programme.




Auditee Feedback Form

Location

Date of Audit

Responder

Strongly Agree

Neither Agree or Disagree

Strongly Disagree

No Basis

1. The audit objectives were clearly communicated to me.

2. The audit took an acceptable amount of time (from entrance to exit).

3. The disruption of daily activities was minimised as much as possible during the audit.

4. My business concerns and perspective were adequately considered during the audit.

5. Communication of audit results and status to me during the audit was timely and adequate.

6. The audit team demonstrated technical proficiency in the audit areas.

7. The audit team demonstrated courtesy, professionalism, and a constructive and positive approach.

8. The audit team’s conclusions were logical and well documented.

9. Audit results were accurately reported and appropriate perspective was provided.

10. The audit report was clearly written and logically organised.

11. Overall the audit provided “value added” to my organisation.

Comments: _____________________________________________________________________________ _____________________________________________________________________________ _____________________________________________________________________________ _____________________________________________________________________________ Source: AlliedSignal Inc.




Periodic Programme Reviews

Audit programmes should keep pace with changes in the business and regulatory climate and cannot be static. Periodically, it pays to step back and take a hard look at the overall programme design and implementation. In these periodic reviews, all elements of the design—objectives, scope, coverage, organisation, resources, and approach—should be critically examined to confirm whether or not they are (still) meeting the needs of the programme’s stakeholders. It is also important to verify that audit teams in the field are actually implementing the programme as designed. There are a variety of mechanisms for conducting these periodic reviews. One possibility is to assemble a team or task force to examine the entire audit process and look for improvement opportunities. The team should probably include representatives of the audited facilities and of upper management, as well as some of the members of the audit programme itself. Another approach that has been used successfully in some companies is to ask the law department or the internal audit department to review the programme. Yet another possibility is to engage an independent third party from outside the company to confirm the adequacy of programme design and implementation versus self-established standards, industry and international standards, and best practices. Regardless of the composition of the reviewing body, a “high performance business” approach may prove useful. In such an approach, the review begins by identifying the key stakeholders of the audit programme and determining/confirming their needs. It is then possible to evaluate the various elements of the audit process to see if it is structured to meet those needs. Once any necessary improvements to the process have been identified, it is possible to assess whether the underlying organisation and resources are adequate for effective implementation.

Appendix A Confirmation Letter




*[Date] *[Name] *[Title] *[Company] *[Address] *[City, State] Re: Environmental, Health, and Safety Compliance Audit Dear __________: Confirming our telephone conversation of *[Date], we will conduct an environmental, health, and safety compliance audit of the *[Facility name] Plant the week of *[Date]. The audit will address air pollution control, water pollution control, spill control and emergency response planning, solid and hazardous waste management, underground storage tanks, soil and groundwater contamination, drinking water management, PCB management, employee safety, loss prevention, and industrial hygiene as well as company policies, guidelines, etc. The audit team will arrive on site on *[Date]. They would like to meet with you and other appropriate personnel on Monday morning to briefly describe the audit and to answer any questions. At that opening conference, it would be helpful if the team could receive a brief description of the current organisation and operations at the facility as well as an orientation tour. Those facility personnel involved with environmental, health, and safety activities will be needed on site for discussions during the audit week. At the conclusion of the audit, the audit team will again meet with you and other appropriate personnel to discuss the team’s findings. This closing conference is a critical part of the audit, and it is very important that you be present.



*[Date] Page 2 *[Name] *[Title] *[Company] The audit team will make its own lodging and transportation arrangements, including cars for use during the week. A dedicated meeting room (work area) is needed at the facility with one large conference table or several smaller tables to accommodate the team members and their working papers. Also, access to a telephone and a copier would be helpful. To assist us in our preparation, we would appreciate having the facility complete the enclosed pre-audit questionnaire (Attachment 1), and provide a copy of as many as possible of the applicable items listed in Attachment 2. The pre-audit documents should be sent to *[The Team Leader] ’s attention by *[Date]. Please do not hesitate to call me if you have any questions about this audit or the Environmental, Health, and Safety Audit Programme in general. Thank you for your assistance in these matters. Sincerely, [Team Leader] Attachments cc:

Letter Attachment 1 Example of Pre-Audit Questionnaire



__________________________________________________________________


I General Information

1. Number of Employees on-site

2. Number of Contractors (Firms) on-

site

3. Number of Contractors’ employees

on-site (daily average)

4. Number of Employees in Company

5. Number of shifts worked

(please specify times)

6. Products

7. Annual production volume

8. Time company has operated at this

site

9. Nature of any previous industrial

activities on this site 1. 2. 3.

10. Name(s) of Doctors and

Occupational Nurses on-site and % time in attendance

1. 2. 3. 4.


__________________________________________________________________


II Locality/Neighbourhood Description

1. Nature of facility location e.g., commercial, industrial, residential, agricultural, rural

2. Are there any schools, hospitals,

nursing homes, prisons, churches or other public buildings within two kilometres of the facility?

3. Are there any nature reserves,

national parks or sites of specific scientific interest within two kilometres of the facility?

4. Distance to nearest industrial

neighbour from the fence line

5. Nature of neighbouring industrial

activities

6. Describe any major hazard

installations within two kilometres of the facility and distance

7. Distance to nearest residential

property

8. Approximate size of the population

near this facility (see table below)

0-1 kilometre 0-100 100-1000 +1000 1-2 kilometres 0-100 100-5000 +5000 2-5 kilometres 0-100 100-10,000 +10,000 9. Number of complaints per year

received from the local community related to HSE issues


__________________________________________________________________


10. Nature of any local community

activities in which the site participates

11. Distance to nearest surface water more than 1 kilometre (please tick) 0.5-1 kilometre adjacent to facility

boundary

within facility 12. Is the facility affected by any natural

hazards, e.g., earthquake, hurricane, flooding?

13. List any off-site facilities

(warehouses, processing units)


__________________________________________________________________


III Policy and Organisation

A Environmental Management System 1. Does the facility have any of its own

specific policies, procedures or guidelines pertaining to:

(a) Health and Safety (b) Environment

Yes No Yes No

List site-specific written HSE policies and procedures below or attach an index if preferred

2. Please provide an organisation chart

showing HSE management structure and responsibilities

3. Who is responsible for identifying

training needs?

4. Who is responsible for developing

and implementing training programmes?


__________________________________________________________________


5. List site specific training programmes/topics below or attach an index if preferred


__________________________________________________________________


IV Functional Areas

A Occupational Environment Who is responsible for occupational

environment programmes at the facility?

Name: Title:

1. Are risk assessments carried out to

evaluate: (a) Internal air quality

(b) Temperature

(c) Lighting levels

(d) Ergonomics

(e) Manual handling

(f) Exposure to hazardous

substances

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

2. Is there a preventative maintenance

programme covering Heating, Ventilation and Air Conditioning (HVAC) and lighting equipment?

Yes No N/A

3. Is there a formalised safety

inspection programme? (a) Who inspects? (b) How often?

Yes No N/A

4. Does the facility have procedures to

report and respond to occupational health hazards?

Yes No N/A

5. How many reports/complaints have

been made in the past two years?


__________________________________________________________________


B Technical Equipment and Machinery Protection Who is responsible for technical

equipment and machinery protection programmes at the facility?

Name: Title:

1. Are risk assessments carried out to

evaluate: (a) Moving parts

(b) Cutting/blades

(c) Rollers and pinch points

(d) Excessive temperatures

(e) High pressure

(f) Other (please specify)

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

2. Is there a preventative maintenance

programme covering maintenance of equipment and machinery, and associated protection?

Yes No N/A

3. Are equipment inspections

conducted to review the safe working condition(s) of the equipment/machinery?

Yes No N/A

4. Are all lifting appliances examined

and certified? Yes No N/A

5. Are all pressure vessels tested and

certified? Yes No N/A

6. Does the facility have procedures for

employees to report faults and for corrective action?

Yes No N/A


__________________________________________________________________


C Materials, Goods and Hazardous Substances Who is responsible for materials,

goods and hazardous substances?

1. Do you maintain an inventory of

hazardous materials used and stored on site?

Yes No N/A

2. Have you conducted risk

assessments on use/storage/transport of hazardous materials, goods and substances?

Yes No N/A

3. Have you a risk reduction

programme, which includes, for example, elimination, reduction of use or personal protective equipment?

Yes No N/A

4. Are Safety Material Data Sheets

giving physical, chemical and toxic properties, and environmental data kept on site for each hazardous material?

Yes No N/A

5. What are the separation distances of

hazardous material areas from the boundary fence and other buildings?

6. Does the facility control the use of

hazardous materials by contractors? Yes No N/A

7. Has the facility undertaken any

renovation or demolition activities in the last 24 months that involved the removal of asbestos?

Yes No N/A

8. Have you completed plans for the

phasing-out and replacement of all CFC’s?

Yes No N/A


__________________________________________________________________________________________________________


11. Please complete the following table or supply copy of internal list: Main Hazardous Materials, Goods and Substances used and stored on-site Type of material¹ Use² Approximate

quantity used per year³

Maximum quantity stored on site

Type of storage4 Storage Area Description5

Spill6 containment type and capacity (m3)

e.g. Solvent Plant degreasing 1000 kg 10 kg AST Covered concrete pad

15m³ with retention basin

Notes: 1 Indicate major hazardous materials used in operations or activities, and include potentially hazardous waste materials 2 Describe use of hazardous material listed, for example, boiler/furnace fuel, degreasing/metal cleaning chemical, wastewater treatment

chemical, etc. 3 Indicate use per year in kilograms or litres 4 Please describe the type of container(s) the material is stored in using one of the following symbols : AST: Aboveground non-buried storage tank D200: 200 Litre drums UST: Buried underground storage tank D25: Approx. 25 litre or smaller O: Other (please specify) e.g., IBC/containers/piles 5 Briefly describe storage area (for example, inside, outside, covered, fenced, locked, restricted access, fireproofed etc.) 6 Indicate type of spill containment provided, if any (for example, retention basin, collection sump, oil/water separator, paved/sealed area,

concrete pad, curbing, bunded area, etc,) and indicate by 'yes' or 'no' if the containment will retain firewater in the event of an emergency.


__________________________________________________________________


D New or Modified Processes Who is responsible for new or

modified processes at the facility? Name: Title:

1. Does the facility have HSE

procedures for change management?

Yes No N/A

2. Have any major new processes

been introduced/installed over the past two years? e.g. new production lines, closing of production lines

If so, what were they?

Yes No N/A

3. Have any major modifications been

made to processes over the past two years?

e.g. machinery replacements If so, what were they?

Yes No N/A


__________________________________________________________________


E Hazardous Work Who is responsible for hazardous

work programmes at the facility? Name: Title:

1. Has the facility conducted formal risk

assessments of hazardous work activities?

(a) Hot work

(b) Confined spaces

(c) Energy isolation work (d) Working at height

(e) Other (please specify)

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

2. Has the facility established

documented procedures for hazardous activities, e.g.

(a) Permit-to-Work/Approval procedures

(b) Pre-use/activity inspections

(c) Issue of personal locks

(d) ‘Buddy’ system ( for lone working)

(e) Engineering controls, e.g guarding

(f) Provision and use of personal protective equipment

(g) Other (please specify)

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A


__________________________________________________________________


3. Are Work Permits/Approvals issued

for: (a) Hot work

(b) Confined spaces/Entry work

(c) Electrical work

(d) Working at heights

(e) Contractor Control

(f) Other (please specify)

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

4. Is specific training provided or

qualifications required for employees involved in hazardous activities?

(Please ensure training topics are included in your answer to section III, question 5.)

Yes No N/A


__________________________________________________________________


F Electrical Safety Who is responsible for electrical

safety programmes at the facility? Name: Title:

1. Has the facility conducted risk

assessments for work on electrically energised equipment and electric circuits?

Yes No N/A

2. What preventative and protective

safety measures are in place, e.g. earthing, fuses and circuit breakers?

3. Is a lock-out/tag-out programme in

place at the facility for electrical and other energy isolation?

Yes No N/A

4. Are regular inspections conducted to

review the safe working condition(s) of the electrical equipment?

Yes No N/A

5. Is specific training provided

to/qualifications required for employees involved in electrical work?

(Please ensure training topics are included in your answer to section III, question 5.)

Yes No N/A


__________________________________________________________________


G Work at Height Who is responsible for working at

height programmes at the facility? Name: Title:

1. Does the facility have personnel or

contractors working at heights? (a) Routinely, as part of their work

activities, e.g. Facilities engineer (b) Non-routinely

Yes No N/A Yes No N/A

2. Has the facility conducted risk

assessments on working at height activities?

Yes No N/A

3. What controls are in place to reduce

risks, e.g. (a) Permits-to-Work

(b) Harnesses

(c) Personal protective equipment

(d) Buddy system

(e) Others (Please specify)

Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

4. Does the facility have an inspection

and maintenance programme for ladders and scaffolding?

Does this include pre-use

inspections, tagging and approvals?



__________________________________________________________________


H Noise Control Who is responsible for noise control

programmes at the facility? Name: Title:

1. What are the main sources of noise

at the facility?

Source

Internal External

e.g. compressors Yes Can be heard outside factory building

e.g. transport movements No Yes

2. Are there any regulatory limits for

noise at the facility? (a) environmental (b) occupational


3. In the last three years, how many

times None

has the facility received complaints from

Less than 5

different neighbours relating to noise?

5 - 10

(please tick) More than 10


__________________________________________________________________


4. Does the facility operate its own

freight transport? Yes No N/A

5. Are there transport movements

before 06.00 and after 22.00 hours? Yes No N/A

6. Are there any loading or unloading

operations before 06.00 and after 22.00 hours?

Yes No N/A


__________________________________________________________________


I Personal Protective Equipment (PPE) Who is responsible for PPE

programmes at the facility? Name: Title:

1. For which activities is PPE required?

2. What types of PPE are provided?

3. Does the facility have an approved

list of PPE and/or an approved list of suppliers?

Yes No N/A

4. Which PPE does the inspection and

maintenance activities cover?

5. Is there a formalised

inspection/maintenance programme?

(a) Who inspects? (b) How often?

Yes No N/A

6. Is training provided to employees

involved requiring PPE? (Please ensure training topics are

included in your answer to section III, question 5.)

Yes No N/A


__________________________________________________________________


J Vehicles and Driving Who is responsible for vehicles and

driving programmes at the facility? Name: Title:

1. Does the facility use (please indicate

approximate numbers): (a) Forklift trucks

(b) Industrial plant (e.g. diggers,

cranes)

(c) Heavy Goods Vehicles (HGV)

(d) Company cars


Owned? Leased? Contractors’?

2. Are there procedures for meeting

designated vehicle specifications? Yes No N/A

3. Is there a formalised

inspection/maintenance programme?

(a) Who inspects? (b) How often?

Yes No N/A

4. Is there a pre-use inspection

procedure for vehicles (other than company cars)?

Yes No N/A

5. Is training provided to employees

driving company vehicles? (Please ensure training topics are

included in your answer to section, III, question 5.)

Yes No N/A


__________________________________________________________________


K First Aid, Medical Examinations and Care Who is responsible for first aid,

medical examinations and care programmes at the facility?

Name: Title:

1. Does the facility have an equipped

medical centre/clinic? If yes, is this staffed: (a) Doctor

(b) Nurse

(c) First Aiders

Yes No N/A 24 hrs Full-time Part-time

24 hrs Full-time Part-time

24 hrs Full-time Part-time

2. How many formally qualified First

Aid providers does the facility have?

3. Where does the facility provide first

aid equipment? (a) Medical Centre

(b) Designated locations

(c) Per First Aider

(d) Security/Gate keepers


Yes No N/A

Yes No N/A

Yes No N/A

Yes No N/A

4. Who inspects and maintains first aid

equipment regularly and how often?

Please attach a summary of accidents/incidents over the past 12 months (if easily available).


__________________________________________________________________


5. Are medical examinations provided

by the facility? (a) Pre-employment

(b) Annual check-ups

(c) Critical situations, e.g.

– exposure to hazardous substances

– pregnancy – return to work after

injury/illness – persons under 18/over 60 – terminal diseases – other (please specify)

Yes No N/A

Yes No N/A

Yes No N/A Yes No N/A Yes No N/A Yes No N/A Yes No N/A

6. Which general welfare and wellness

programmes are provided for employees?


__________________________________________________________________


L Fire Protection and Control Who is responsible for fire protection

and control programmes at the facility?

Name: Title:

1. Is this facility regulated in relation to

flammable/explosive substances and/or for fire protection?

If so, what

certificates/permits/licences does the facility hold?

Yes No N/A

2. Does the facility have any areas

designated as explosion hazard zones?

Yes No N/A

3. Has the facility conducted a risk

assessment of ignition, fire and explosion sources? e.g. through insurance inspections

Yes No N/A

4. Has the facility installed fire

protection equipment: (a) Fire detection (heat or smoke)?

(b) Alarms?

(c) Sprinklers or other dousing

systems?

(d) Other (please specify)?

Yes No N/A

Yes No N/A

Yes No N/A

5. Does the facility have an emergency

and evacuation plan? Yes No N/A


__________________________________________________________________


6. Does the facility provide instruction,

training, and testing of emergency plans?

Please specify

Yes No N/A

7. When was the last drill carried out?

Date:


__________________________________________________________________


L Fire Protection and Control (continued) 8. Does the facility have a trained

emergency response team or nominated fire marshals?

Please specify

Yes No N/A

9. Has the facility experienced any

explosions or fires over the past: (a) ten years

(b) five years

(c) two years

Yes No N/A

Yes No N/A

Yes No N/A

10. Has the facility experienced any

near-misses involving explosions or fires over the past:

(a) ten years

(b) five years

(c) two years

Yes No N/A

Yes No N/A

Yes No N/A

11. Of these, how many have been

reportable to a government agency


__________________________________________________________________


M Air Pollution Control Who is responsible for air pollution

control programmes at the facility? Name: Title:

1. Has the facility identified and

documented all sources of air emissions?

Yes No N/A

2. Has the facility identified the nature

and quantities of pollutants emitted to atmosphere?

Yes No N/A

3. Which air emissions from the facility

are regulated by authorities?

4. Which air pollutants emitted from the

facility are required to be monitored by a government agency? If none, state ‘none’

5. Which types of air pollution control

equipment are installed, e.g. scrubbers, dust filters?

6. Does the plant operate planned

maintenance procedures for its air pollution control equipment?

Yes No N/A

7. State average number of public

complaints for last three years attributed to air emissions from the facility, e.g. odour, dust, VOCs, smoke


__________________________________________________________________


8. How many cooling towers or static

water tanks are located in the facility?

9. Are 6-monthly chlorinations carried

out on these to prevent growth of legionella pneumophilia?

Yes No N/A

10. Does the facility dispense fuel to

motor vehicles? Yes No N/A


__________________________________________________________________


N Water Pollution Prevention Who is responsible for water

pollution prevention programmes at the facility?

Name: Title:

1. Does the facility have any targets or

programmes for reducing water consumption?

Please specify

Yes No N/A

2. Are waste water discharges

regulated by a permit or consent? Yes No N/A

3. Does the facility conduct any effluent

monitoring? Yes No N/A

4. Does the facility make use of an on-

site wastewater treatment system prior to effluent discharge?

Yes No N/A

5. If the facility has own wastewater

treatment plant, how is the sludge disposed of

6. Is any process wastewater recycled? Yes No N/A

7. Does any portion of the facility’s

drinking water supply come from on-site wells or surface water sources?

Yes No N/A


__________________________________________________________________


O Waste Management Who is responsible for waste

management programmes at the facility?

Name: Title:

1. Does the facility generate wastes

that are defined as ‘hazardous’ or ‘special’ under government regulations? Please provide details in the Table below (question 6).

Yes No N/A

2. Does the facility require/have a

permit or license for its waste activities?

Yes No N/A

3. Does waste treatment and/or

disposal take place/ever taken place on-site?

Please specify

Yes No N/A

4. Are any waste materials separated

and sent for recycling? Please provide details in the Table below (question 6).

Yes No N/A

5. Does the facility monitor off-site

disposal activities? Yes No N/A


__________________________________________________________________________________________________________


6. Please complete the following table. Solid and Hazardous Waste Generation, Treatment and Disposal

Description of Waste

Generation

Process1 Classification e.g. hazardous,

liquid, inert

Estimate of Annual Quantity Generated (tons or Kg/year) (if available)

Method of Treatment and final Disposal2

Waste minimisation target and date (if

any) e.g. cardboard packaging

raw materials/ supplies

inert 6 tons Re/L 20% reduction by 1999

Notes : 1 Indicate which type of equipment or operation generates this waste stream. 2 Please enter one of the following letters as appropriate. If the disposal or treatment is on-site, please circle the letter.

Re = Recycled externally S = Sold for further use (please specify) T = chemical or physical Treatment

I = Incinerated

Ri = Recycled Internally L = Landfill O = Other (please specify

Letter Attachment 1 Example of Pre-Audit Questionnaire __________________________________________________________________


P Soil and Groundwater Protection Who is responsible for soil and

groundwater protection programmes at the facility?

Name: Title:

1. Have you conducted a survey to

identify actual areas of soil and groundwater contamination resulting from previous industrial practices or activities?

Yes No N/A

2. Has there been any remediation of

land which is contaminated? Yes No N/A

3. Does the facility have any

underground (buried) tanks in or out of service with associated piping? (See also 7)

Yes No N/A

4. Does the site have procedures to

protect soil and groundwater (e.g. spill prevention and containment programmes/facilities) and a written spill reporting procedure?

Yes No N/A

5. Does the stormwater drainage

system include any interceptors? Yes No N/A

6. Could excessive stormwater cause

secondary containment, interceptors etc. to flood and overflow?

Yes No N/A



7. Please complete the following table. Bulk Storage Tanks Tank Volume

Above ground AST or Buried UST

In/Out of service

Material stored (or previously stored)

Leak detection and/or release prevention

Tank age (years)

Leak test

e.g 50m³ UST In fuel oil none 35 none

Notes: 1 Indicate volume of tank in litres (I) or cubic metres (m³) 2 Indicate whether tank is above ground - use letters AST, or below ground - use

letters UST 3 Indicate whether the tank is equipped with leak detection measures, overfill

protection, or corrosion protection 4 Indicate date of most recent leak test (including hydrostatic pressure testing,

sampling and analysis of surrounding soil, etc.). If not tested, indicate ‘None’.



Q Product Stewardship Who is responsible for product

stewardship programmes at the facility?

Name: Title:

1. Has the facility determined the

environmental impacts of all its product and packaging materials, including transport packaging materials?

Yes No N/A

2. Are environmental impacts formally

considered during development and marketing of products?

Yes No N/A

3. Do raw material specifications

routinely include HSE requirements? Yes No N/A

4. Do manufacturing machinery

specifications routinely include HSE requirements?

Yes No N/A



R Energy Conservation Who is responsible for energy

conservation programmes at the facility?

Name: Title:

1. Has the facility carried out a

systematic review of all its uses of energy, to identify major energy consuming equipment or activities?

Yes No N/A

2. Have energy conservation objectives

and targets been set? Please specify

Yes No N/A

3. Has the facility defined Energy

Accountable Centres (i.e., discrete operating units within the site for which energy consumption figures are available)?

Yes No N/A

Letter Attachment 2 Pre-Audit Information Request


• Facility plot plan or map • Directions to the facility • Visitor safety requirements (e.g., personal protective

equipment, orientation, specialised training, special clearances, etc.)

• Completed pre-audit questionnaire • Description of the facility’s operations/processes • Facility organisation chart, showing HSE responsibilities • Local laws, regulations, and ordinances related to the scope

of the audit • List of current environmental licenses, certificates, and

authorisations • Copies of permits for wastewater discharges and example air

emission permits • Recent regulatory agency inspection/enforcement

correspondence • Recent internal and intra-company environmental, health, and

safety audit reports • Table of contents for facility-specific environmental, health,

and safety policies and procedures

Appendix B Roles and Responsibilities of the Audit Team


Appendix B Roles and Responsibilities of the Audit Team - Audit Team Leader Responsibilities



• Select team members and assign audit responsibilities. • Gather and distribute background information. • Identify applicable federal/national, state/provincial, and local

regulations and company policies and procedures. • Conduct advance visit to the facility (if necessary). • Review and revise audit strategy and assigned duties as

necessary. • Determine and confirm arrangements with the team members

and the facility: − Travel arrangements − Hotel/travel reservations

• Prepare items for audit (forms, supplies, protocols). • Coordinate pre-audit team meeting(s). On-Site Activities

• Lead opening meeting presentation. • Serve as liaison between team and facility personnel to

ensure that all team members are appropriately scheduled to meet with facility personnel.

• Solicit feedback from each team member on the status of

work accomplished throughout the audit. • Perform audit duties as determined by the audit strategy.

Appendix B Roles and Responsibilities of the Audit Team - Audit Team Leader Responsibilities


• Review assigned protocol steps with each auditor to ensure that all steps are covered appropriately.

• Document the rationale for changing the scope of the audit (if

necessary). • Understand the context for and meaning of each finding

reported by the team. • Provide periodic feedback to facility personnel on the status of

the audit and the findings of the team. • Prepare the exit meeting discussion sheets listing all findings

as summarised by the team. • Ensure that all exit meeting discussion sheets are reviewed by

each team member. • Review all findings with key facility personnel prior to the exit

meeting to ensure the accuracy of all findings. • Lead presentation of exit meeting discussion. • Summarise reporting schedule and format. Post-Audit Activities

• Review all working papers to ensure that all topics were covered and that all findings are corroborated by working paper notes.

• Prepare draft report. • Distribute draft report for comments. • Incorporate comments where appropriate into the final report.

Appendix B Roles and Responsibilities of the Audit Team - Audit Team Member Responsibilities



• Make travel arrangements (if required). • Attend pre-audit team meeting (if required). • Prepare for the audit by reviewing appropriate

federal/national, state/provincial, and local regulations, company policies and procedures, and available background information.

• Modify or annotate the protocol to reflect facility-specific

requirements, state and local regulations, and information gained during review of background information.

On-Site Activities

• Perform duties assigned by the team leader during the audit. • Serve as a resource for other audit team members during the

audit. • Report on progress to the team leader throughout the audit,

including any problems encountered. • Share observations/concerns with other team members during

the audit to ensure that each is addressed appropriately. • Keep facility personnel apprised of findings as they are noted. • Summarise all findings and report them to the team leader

before the close-out meeting. • Assist with preparing the exit meeting discussion sheets.

Appendix B Roles and Responsibilities of the Audit Team - Audit Team Member Responsibilities


• Ensure that all findings noted in your working papers are presented on the exit meeting discussion sheets and accurately reflect the facts as you understand them.

• Contribute during the exit meeting when questions are raised

about your findings. Post-Audit Activities

• Review draft audit reports for: − Wording changes − Suggested changes in placement of findings within the

report • Provide input as necessary when findings in the draft report

are challenged.

Appendix C Guide to Some Health, Safety and Environmental Acronyms


Appendix C Guide to Some Health, Safety and Environmental Acronyms


AAQS Ambient Air Quality Standards NSR New Source Review ACGIH American Conference of Governmental Industrial

Hygienists OCAW Oil, Chemical, and Atomic Workers

AHERA Asbestos Hazard Emergency Response Act of 1986 (Title II of TSCA)

OEL Occupational Exposure Limit

ANSI American National Standards Institute OSHA Occupational Safety and Health Administration ARAR Applicable or Relevant and Appropriate

Requirements OTA Office of Technology Assessment

AQCR Air Quality Control Region PEL Permissible Exposure Limit (for workplace) ASHRAE American Society of Heating, Refrigeration, and Air

Conditioning Engineers PCBs Polychlorinated Biphenyls

BACT Best Available Control Technology pH Measure of acidity or alkalinity BOD Biochemical Oxygen Demand PHA Process Hazard Analysis BPCT Best Practicable Control Technology P&ID Piping and Instrumentation Design BPT Best Practical Treatment PM Particulate Matter (in air) BTU British Thermal Unit PM10 PM with <10 micron diameter; respirable CAA Clean Air Act PMN Premanufacture Notification (under TSCA) CAAA Clean Air Act Amendments POTW Publicly Owned Treatment Works CAS Chemical Abstracts Service Registration Number PRPs Potentially Responsible Parties (under CERCLA CAG EPA’s Carcinogen Assessment Group PSD Prevention of Significant Deterioration CAIR Comprehensive Assessment Information Rule (under

TSCA) PSM Process Safety Management

CERCLA Comprehensive Environmental Response, Compensation, and Liability Act (The Superfund Law)

RACT Reasonably Available Control Technology

CFCs Chlorofluorocarbons RCRA Resource Conservation and Recovery Act CFR Code of Federal Regulations RQ Reportable Quantity (for spill reporting) CMA Chemical Manufacturers Association RTECS Registry of Toxic Effects of Chemical Substances CO Carbon Monoxide SARA Superfund Amendments and Reauthorisation Act of

1987 COD Chemical Oxygen Demand SDWA Safe Drinking Water Act CPSC Consumer Product Safety Commission SERC State Emergency Response Commission CWA Clean Water Act SIC Standard Industrial Classification DMR Discharge Monitoring Report SIP State Implementation Plan DOT Department of Transportation SNUR Significant New Use Rule EPA Environmental Protection Agency SPCC Plan Spill Prevention Control and Countermeasures Plan EPCRA Emergency Planning and Community Right-to-Know

Act (Title III of SARA, commonly called Right-to-Know or SARA Title III)

SQG Small Quantity Generator (of hazardous waste)

FIFRA Federal Insecticide, Fungicide, and Rodenticide Act STS Standard Threshold Shift GEP Good Engineering Practice SWDA Solid Waste Disposal Act HAZOP Hazard and Operability Study TCLP Toxicity Characteristic Leaching Procedure (under

RCRA) Hazwoper Hazardous Waste Operations and Emergency

Response Title III Emergency Planning and Community Right-to-Know

Act HCS OSHA Hazard Communication Standard (Worker

Right-to-Know) TLV Threshold Limit Values (for workplace exposure)

HMTA Hazardous Materials Transportation Act TPQ Threshold Planning Quantity (for emergency planning HSWA Hazardous and Solid Waste Amendments (1984

RCRA Amendments) TSCA Toxic Substances Control Act

IARC Internal Agency for Research on Cancer TSDF Treatment, Storage, and Disposal Facility IDLH Immediate Danger of Life and Heath TSP Total Suspended Particulate (in air) IH Industrial Hygienist TSS Total Suspended Solids (in water) LAER Lowest Achievable Emission Rate UST Underground Storage Tank LEPC Local Emergency Planning Committee VHAP Volatile Hazardous Air Pollutant LOTO Lockout/Tagout (Control of Hazardous Energy) VOC Volatile Organic Compound MACT Maximum Available Control Technology WWTP Wastewater Treatment Plant MSHA Mine Safety and Health Administration Z List OSHA list of hazardous chemicals (29 CFR 1910

Subpart Z, Worker Right-to-Know) MSDS Material Safety Data Sheet NAAQS National Ambient Air Quality Standards NEPA National Environmental Policy Act NESHAP National Emission Standards for Hazardous Air

Pollutants

NIOSH National Institute for Occupational Safety and Health NOAA National Oceanic and Atmospheric Administration NOx A mixture of nitrous oxide and nitrogen dioxide NRC National Response Center NPDES National Pollutant Discharge Elimination System

NSPS New Source Performance Standards

audit Skills Training Handbook

Documents

Transcript of audit Skills Training Handbook