Pitfalls of Testing and Commissioning

Pitfalls in Data

Centre Testing and

Commissioning

Two Projects Compared PROJECT A PROJECT B

Data Centre Handed over with minor defects to be rectified during DLP

Data Centre Handed over with very large list of outstanding defects to be addressed during User fit out stage and DLP

Completed on time 4 weeks past contract completion date

Commissioning runs smoothly Major discussion argument and some conflict during commissioning.

Commissioning proceeded to plan Incidents included 3 fires one flooding power lost to adjacent area of the site.

Design concepts validated by commissioning process

Urgent re-design of some systems needed to correct design flaws.

Major Equipment deliveries received on time Major Equipment deliveries delayed requiring re-scheduling of the construction program.

Operations and maintenance manuals handed over on time to Future Facility Managers

Manuals rejected and undergoing review and resubmission

WHICH ONE IS YOUR PROJECT ?

Which one represents Successful Commissioning?

TWO PROJECTS

Project 1 - Successful

Management happy

Contractors happy

Construction team congratulated

BUT

Experience shows commissioning often throws

up unexpected problems

How thorough was the actual commissioning?

TWO PROJECTS

Project 2 also successful

Program allowed float between contractor hand over and occupancy of the Data Centre

Potential Operational problems were identified AND RECTIFIED before Data Centre became

operational.

Operations staff now know exactly how the Data centre will perform

When finally received the Manuals are known to be accurate.

COMMISSIONING Now that you have your new data centre

designed and under construction how do you

know it will perform as you need it to?

COMMISSIONING is your opportunity to see

how your Data Centre will perform under

extreme conditions

MAKE THE MOST OF THIS OPPORTUNITY

YOU PROBABLY WONT GET ANOTHER ONE

A Commissioning Model

What is the appropriate model for data centre

commissioning?

One accepted regime is the:

American Society of Heating Refrigeration and

Air Conditioning Engineers (ASHRAE) model

for Commissioning

A Commissioning Model

ASHRAE have identified five categories of

Formal Commissioning Activities.

Level 1 Factory Acceptance Test

Level 2 Field Component Verification

Level 3 System Construction Verification

Level 4 Site Acceptance Testing

Level 5 Integrated Systems Testing

Shop drawings

Not highlighted by ASHRAE but is critical to a

successful project.

The shop drawings are the foundation that a

successful commissioning is built on.

This is the first opportunity to ensure the suppliers

have got it.

ASHRAE Level 1 Factory acceptance testing

Often abbreviated as FAT

Also referred to as Factory Witness Test when the test is performed in the presence of an owners representative.

This test primarily is product testing

To ensure compliance with Manufacturers advertised specifications, ratings and characteristics.

For many items of equipment these activities come under the manufacturers QA systems.

Testing takes the form of Type testing

Sample testing

Often the testing is mandated or covered by Australian or International Standards

It is common to request copies of the test reports in

the contracts for minor items


Major plant items are often, and should be, witness

tested (generators, UPS, switchboards, chillers etc.)

Manufacturers standard tests may not be adequate

Tests often need to be augmented by owner-defined functional performance or aesthetic requirements.

Attendance can be costly particularly when

equipment is manufactured overseas. Must be included in the budget.

Dont treat this as an overseas junket!


ASHRAE Level 2 Field component verification

This is essentially a contractor activity and is generally covered

under the contactors quality Assurance purchasing activities.

It includes inspection or testing of products and goods upon

delivery to site to ensure that:

The products delivered match those purchased and tested during factory acceptance

Products match the order

Goods delivered have not been damaged during shipment

Goods delivered have not been altered

Goods delivered are these witnessed during FAT

ASHRAE Level 2 Field component verification

As an end user you do not directly participate this

level of testing but must ensure that it does occur.

You can influence, often unintentionally and

adversely, this level of commissioning via:

Demanding construction program

Contract requirements

Cash flow requirements - EOFY or other contract payment dates

ASHRAE Level 3 System Construction verification

Includes field inspections and certifications that the components are assembled into systems as required by the

plans and specification.

This typically is the defects inspections and certification stage of a project that is generally well understood within

the construction industry.

In many standard construction projects this is the highest level of commissioning that is carried out as part of the final

completion and certification of a construction project.


The builders and consultants are generally comfortable and familiar with their roles at this level.

They also will generally fall into the standard practices and routines that they are familiar with.

As a client you may not be as familiar with these standard practices

These standard acceptance practices

generally wont be rigorous enough for a high reliability installation such as a data centre.


It is important to understand the roles that are played

out here The designers generally certify the designs

The installers certify that they believe they have built the installation to the design

The building hand over process relies on these certifications.

The project management team ensure and certify that the work is complete and in accordance with the Contract. The designers often have a

role in this inspection regime.

The important word here is contract. You will get no more and no less than what the contract asks for.

ASHRAE Level 4 Site acceptance testing

Most often referred to as SAT

SAT includes demonstration that related components

equipment and ancillaries of a defined system

operate and function to the acceptance criteria.

This level of commissioning is often only performed

on the more critical systems including: Mechanical Services

Electrical Services

Fire Services

Communications (sometimes Level 3)


It should include:

Normal modes of operation

Maintenance mode of operation

Checking safeties

Checking monitoring and control functions

Checking performance


The construction and Project Management Teams

are comfortable and familiar with their roles at this

level.

They also will generally fall into the standard practices and routines that they are familiar with.

This activity usually falls within the agreed contract completion time.

Level 4 testing can be rushed if the construction falls behind program.

The various parties involved are focused on their particular

systems and tend to commission and test in isolation of the

other systems.


THIS IS THE FINAL STAGE FOR

MOST PROJECTS INCLUDING

MANY DATA CENTRES

ASHRAE Level 5 Integrated system testing

The Functional performance testing of interrelated components and

systems to respond as intended to expected and unexpected anomalies.

There is no industry recognized formula for this level

This level of testing is expensive

This level of testing goes beyond demonstrating contract compliance

This level of testing can, and probably will, expose design problems

This is one way that design and construction of a complex high

reliability systems such as Data Centre Infrastructure can be

validated


Why perform level 5 testing?

Testing to level 4 only tests the systems in isolation.

Testing to level 4 is often performed with some systems incomplete.

Testing to level 4 will only demonstrate that the systems conform to the design, it does not validate the design.

Are the systems properly integrated? Will the generating set support the UPS?

Does the monitoring system properly communicate with and record alarms for the critical sub-system?

Allows operation and maintenance personnel to learn how the system operate and interact.


For Level 5 testing to be successful it needs

Careful planning

Good programming

The right participants

For Level 5 Commissioning to be successful you need buy in

by all parties. You need to allocate:

Time

Money

Resources

Manuals

Also not highlighted by ASHRAE but is critical to

successful operation of a data centre.

Accurate manuals are required to ensure trouble free

operation and timely maintenance of the data centre.

Incorrect manuals and as-built drawings can have

disastrous consequences years after completion .

Commissioning Overview

Two Projects Compared PROJECT A PROJECT B

Data Centre Handed over with minor defects to be rectified during DLP

Data Centre Handed over with very large list of outstanding defects to be addressed during User fit out stage and DLP

Completed on time 4 weeks past contract completion date

Commissioning runs smoothly Major discussion argument and some conflict during commissioning.

Commissioning proceeded to plan Incidents included 3 fires one flooding power lost to adjacent area of the site.

Design concepts validated by commissioning process

Urgent re-design of some systems needed to correct design flaws.

Major Equipment deliveries received on time Major Equipment deliveries delayed requiring re-scheduling of the construction program.

Operations and maintenance manuals handed over on time to Future Facility Managers

Manuals rejected and undergoing review and resubmission

WHICH ONE IS YOUR PROJECT ?

Which one represents Successful Commissioning

Commissioning Overview LEVEL 1 FACTORY ACCEPTANCE


What to look for: Basic operation almost always okay

What is missing usually interface with other services

Conformance with Australian Standards Particularly relevant for overseas suppliers

Rigorousness of FAT regime

Review of non-witnessed test results prior to despatch


Will the equipment fit?

Commissioning Overview LEVEL 2 COMPONENT VERIFICATION

Beware of negative influences by artificial contract

dates!

Commissioning Overview LEVEL 3 SYSTEM VERIFICATION

This is a contractor activity that is determined by the

construction contract

Once the contracts are in place you will have very

little opportunity to influence it

Building works predominantly complete

Requires co-ordination between trades to avoid

abortive works

Commissioning Overview LEVEL 3 SYSTEM VERIFICATION

SAFETY

Commissioning Overview LEVEL 4 SAT

These tests are defined during the original contract.

You can influence Level 4 commissioning by ensuring

that the specifications and contracts clearly spell out

the acceptance testing. Cost

Program

REMEMBER THIS LEVEL OF COMMISSIONING

CONFIRMS THAT THE INSTALLATION MEETS THE

ORIGINAL DESIGN.

Commissioning Overview LEVEL 4 SAT

Systems MUST be complete

Commissioning Overview LEVEL 5 Integrated System Testing

The decision on performing IST is totally in the

customers hands

It will probably be the only opportunity to observe how

the data centre will respond under extreme stress

Not industry standard


Pifalls Contractors not on board or interested in IST consider moving IST

outside contract completion date

Testing before the data centre is complete

Monitoring system must be complete

Who does the ITPs? Must be someone who understands the design

Takes a lot of time

Contractors are typically only concerned with completing construction to the design documents

Designers often only test scenarios they have designed for

Consider peer reviewer or independent commissioning agent but get them involved early they need to have time to assess and understand the design


Pifalls Load in the data hall

How big a load

How many load banks

Consider many small load banks to avoid heat damage and fire system activation

Issues identified

KEY POINTS

PLANNING

Decide on your commissioning needs

Level 1 involvement

Level 5

Allow sufficient Budget

Allow time for commissioning

KEY POINTS

PEOPLE

Involve the Designers

Involve the user/owners

Involve the facility managers

Consider an independent commissioning agent

Make sure the team has

Skills

Experience

Collaboration

KEY POINTS

DOCUMENTATION

Contract requirements

Define responsibilities

Workshop Drawings

Manuals

The Testing Script (ITPs)

KEY POINTS

RELATIONSHIPS

Collaboration

Communication

Expectations

Everyone must be engaged

KEY POINTS

COMPLETION

Works must be complete

and

Previous Level Testing must

be Complete

Questions

Pitfalls of Testing and Commissioning

Documents

Transcript of Pitfalls of Testing and Commissioning