Example of Electrical Utility Problem Investigation using 5-why methode

Investigation Of Mutiara Genset Trouble By: Maintenance Engineering – VICO Indonesia

1. Background Started by blackout of Mutiara Central Plant on October 30, 2007, Mutiara gensets was

then frequently trip. We have recorded 8 trip events on Genset#5, 10 trip events on

Genset#3 and 2 trip events on Genset#4. At the last trip of Genset#4, found that the

winding indicate poor insulation resistance leading to refurbishment of this unit (by

Hidup Baru).

The lost of production regarding this problem is the major concern.

To know why is important to eliminate and anticipate the same problem in the future, at

the end to safe our production.

Let’s say MTZ-1900A as genset#2; MTZ-1900B as genset#3; MTZ-1900C as genset#5;

MTZ-1900D as genset#4 and backup Diesel Genset as genset#1.

2. Chronologies 1. Trip (Blackout) on Oct 30, 2007 at 03:06 AM

Normal running : Genset#2,3,4 and 5

- Genset#5 Trip indicated Loss of Excitation

- Genset#3 Trip indicated Reactive Power Trip

- Genset#4 Trip indicated under frequency

- Genset#2 Trip indicated nothing, guess it was under frequency

Load shedding system worked, but the response to shed maybe slower than the

overload sensed unit#2 and #4

2. Trip on 31 Oct, 2007 at 01:22 AM


- Genset#5 Trip indicated Phase over current

- Genset#4 engine S/D due to detonation

Genset#1 was successful to online so the system backed up

Load shedding worked as expected.

3. Trip on Nov 03, 2007 at 01:20 AM


- Genset#5 Trip indicated Phase Overcurrent

- Genset#4 Trip indicated Phase Overcurrent and Stator RTD over temp

- Genset#3 Trip no indication recorded in GMR

Load shedding worked as expected

4. Trip on Nov 07, 2007 at 23:09 PM

Normal running : Genset#1,2,3 and 5. Genset#4 was being repaired at Hidup Baru.



Load Shedding system worked as expected

5. Trip on Nov 10, 2007 at 14:22 PM and 17:25 PM





6. Trip on Nov 14, 2007 at 00:24 AM





7. Trip on Nov 16, 2007 at 11:56 AM





8. Trip on Nov 23, 2007 at 01:06 AM





3. Description of incidents

At the first of occurrence of this problem is reported by field maintenance that on 30th

of

October 2007 Mutiara Central Power black out. The normal condition before S/D was

genset#2,3,4 and 5 run in parallel. Genset#5 was trip at the first time with indication Loss

of Excitation. Genset#3 then (almost simultaneously) trip with indication Reactive

Power. After that, genset#4 S/D with indication underfrequency due to suddenly

overloaded. This is because genset#1 fail to run for backup the loss of genset#3 and 5.

This condition had lead to blackout of MCP. Under this circumstance, the investigation

was then focused to genset#3 and genset#5.

The second, on 31st of Oct 2007, the event shown with phase overcurrent in genset#5 and

genset#4 S/D due to detonation (fuel problem, found the fuel solenoid abnormal

operation). Genset#1 was successfully backup the lost of Genset#5, therefore no blackout

on this event. The normal condition before S/D was genset#2,3,4 and 5 run in parallel.

After this event, maintenance team did corrective action to:

- Adjust fuel regulator differential pressure

- Repair fuel injection solenoid.

- Adjust AVR genset#2 and replace varistor (part of exciter).

The third, on 3rd

of Nov 2007. The normal condition before was genset#2,3,4 and 5 run in

parallel. The event was almost the same as before, but, in addition, genset#4 was also trip

phase OC and RTD over temperature and genset#3 trip with no indication. Why no

indication? This is because the memory of GMR was full by alarm of RTD event.

Because maintenance crew install a dummy RTD without disabling the protection

function.

The fourth and next event is almost the same condition, but the most mysterious

problems to find the root causes. The symptoms are genset#5 trip with indication of Loss

of Excitation and Genset#3 with Reactive Power indication. This is almost occurred in

every event since the first blackout except on event when the phase over current existed.

4. Finding

• Informed that there was an AVR replacement work in unit#2 at 29th

of Oct 2007.

And after trip on 31st Oct 2007, a report from maintenance crew said that AVR of

unit#2 is re-adjusted.

• At the event of phase overcurrent on 31st of Oct and 03

rd of Nov, found that an

excessive of reactive power produced by generator while the active power is

considered in normal range.

• From correspondence with operators informed that power factor hunting at the

moment just before trip happened.

• Found LoE setting of unit#5 and Reactive Power protection setting of unit#3 are

too sensitive.

• At the last event on November 23, 2007, informed that genset#3 and genset#5 trip

with the same indication as before while attempting to parallel genset#1. The

AVR of genset#1 at this occurrence was still in problem.

5. Analysis

By using five-why analysis, let’s define the real causes of the problem.

This is not as simple root cause analysis as we imagine, because many events associated

in this investigation.

Possibility of causes

If we see the events chronology, we can say that the whole events show two different

behaviors. The phase over current and LoE accompanied with excessive reactive power.

But let’s see on “Finding” that the over current event was driven by excessive reactive

power also. This is significantly valuable information! Then, we can say that the over

current symptom is really the excessive reactive power symptoms. We will then consider

the excessive reactive power mainly in our discussion since this symptom happened in

every event.

The Loss of Excitation event basically occurred when a machine loss the field current

produced by exciter or exciter didn’t produce any power. The relay sensed the ‘loss of

excitation’ by seeing the generator impedance from its terminal or by monitoring the

reactive power flow. When reverse reactive powers reach a value limited by the setting

the relay would sense as ‘loss of excitation’. When multiple machines operated in

parallel, a really occurrence of loss of excitation in one machine will cause that machine

to get its excitation from system by absorb reactive power from system to maintain its

terminal voltage constant as the system voltage. In the real loss of excitation problem, we

would see a machine has reverse reactive power (or indicated by lead power factor at cos

phi meter) while the other machines will give more reactive power than normal. In these

events we see that two machines (genset#3 and genset#5) had reverse reactive power.

The fact that informed by operator regarding power factor hunting indicated on cos phi

meter is also strengthen this hypothesis. We then consider by this information that the

Loss of Excitation was only a nuisance trip caused by reactive power or power factor

hunting.

How can Reactive Power swing?

In the parallel operating system, the voltage and speed of the machines is considered as

constant, so that if any disturbance in the system occurred the voltage would drastically

dip, the AVR would response to stabilize the voltage to its system voltage reference.

The excitation system will increase or decrease the field current to maintain the voltage

constant. This process instantly, cause reactive power varies increase or decrease. The

small vary would be tolerable. But, in most system uses induction motors the frequent

event of motor starting and any fault in system would tend to a serious consideration due

to sudden voltage drop that might be happen frequently. In this circumstance, the role of

AVR is very important. The stability of AVR and the precise droop setting in parallel

operating is a curious matter. Once a AVR unstable to response the system changes, it

would affect the others on the parallel operation, cause the power factor (indeed reactive

power) swing or usually called “hunting” among the units.

Let’s start to analyze symptoms we have found in event log.

5.1. Event 30th

of Oct 2007 The normal condition was genset#2,3,4 and 5 run in parallel. Genset#5 was trip at the

first time with indication Loss of Excitation. Genset#3 then (almost simultaneously) trip

with indication Reactive Power. After that, genset#2 and 4 S/D with indication

underfrequency due to suddenly overloaded. This is because genset#1 fail to run for

backup the loss of genset#3 and 5. This condition had lead to blackout of MCP

The event sequence is as follow:

Genset#5

Trip LoE

Genset#3 Trip

Reactive Power

Genset#2&4 Trip

Underfrequency

BLACKOUT

Genset#1 Attempt to auto-

start and online Genset#1 fail to online

This two events almost simultenously

We identify 4 critical points.

The five-why analysis is as follow:

The red box indicated the critical points to be evaluated. The brown boxes are the root

causes found as result of this analysis.

The root causes of event 1 are AVR genset#2 problem and poor setting of the relay. This

guess to AVR genset#2 is strengthened by information of adjustment this unit after trip

on 31st Oct 2007, due to power factor hunting. See Appendix1.

5.2. Event 31st Oct and 03

rd Nov 2007

The sequence of event and the five-why analysis is as follow:

MCP Power System Blackout

1ST

Why?

Could not synchronize to

system

2nd

Why?

Genset#5

Trip LoE

Genset#3 Trip

Reactive Power

Genset#1 fail to online Genset#2&4 Trip

Underfrequency

Unstable frequency

Unstable voltage

Governor problem

AVR problem Sync Check relay

problem

Component damage

Improper tuning

3rd

Why?

4th

Why?

5th

Why? 5

thWhy?

True Loss of Field or exciter

Power Factor unstable/hunting then

reach the relay pickup setting value

2nd

Why?

Field winding shorted

Excitation current varies

AVR sense voltage

variations

Root causes:

-Frequent system fault/disturbances

-AVR genset#2 problem�unstable

-Lack of generator data leading to

mistake in relay setting�too

sensitive?

3rd

Why?

4th

Why?

5th

Why?

Suddenly Overloaded

Load Shed not

working properly

Response of relay to trip

faster than response of

Load Shedding system to

shed load as required

Time delay setting of UF

relay too short, as found

1s. Typically 5s in

common practices.

3rd

Why?

4th

Why?

2nd

Why?

The basic cause of the engine detonation seems tend to mechanical problem regarding the

fuel system.

The basic cause of overcurrent problem in genset#4 and 5 is the contamination of dirt

and moisture to degrade the genset#4 winding.

5.3. Event on 07th

Nov 2007 and after. In this event, genset#4 was out of service due to poor insulation finding. The normal

condition was genset#1,2,3 and 5 online. Actually several trip events occurred within

Genset#5, #4 Trip Phase

overcurrent

Genset#4 RTD over temp

Genset#4 Engine S/D

due to detonation

NO

BLACKOUT

Genset#1 Attempt to auto-

start and online

Genset#1 succesful to online

2nd

Why?

Supply excessive KVAR while KW

remains in normal value

3rd

Why?

Fault or disturbances in the

system

4th

Why?

Found genset#4 winding was

poor insulation � fault

occurs if voltage applied to

5th

Why?

Root cause: Contamination of moisture and dirt

because no filter in air cooling inlet.

2nd

Why?

Found regulator diff pressure setting too high

fuel injection system abnormal operation

3rd

Why?

Fuel quality degraded

Solenoid fuel injection abnormal operation

4th

Why?

Broken solenoid, loss electrical control

Fuel contaminated, poor gas quality

5th

Why?

Root Causes: Lifetime of solenoid elapsed

Does the PM checklist ask to check this solenoid??

Bad quality gas from well

Genset#4 Engine

S/D detonation

Genset#5 and 4 Trip Phase

Overcurrent and RTD overtemp

Phase current increase to level beyond

the setpoint value (414A).

The trip value as recorded was 430 A

Fuel system problem

1ST Why?

1ST Why?

these events, but all of with same indication, Loss Excitation of Genset#5 and Reactive

Power on genset#3. Several maintenance actions have been done during these events. But

the same problem still existed. Since the first problem had the same symptom, and the

root cause analysis say that the problem is AVR, then we must focused to find the AVR

problem. Which AVR was got problem?

The last event on 23rd

November 2007 provided valuable information. Genset#1 got a

problem with the voltage regulator. This problem was identified by the fact of

uncontrollable overvoltage condition in genset#1 causing this genset out of system.

Because this genset is for backup the power during absence of genset#4, then genset#1

tried to re-tie to system after few observations the voltage was stable. Instantly, genset#3

and 5 tripped with same indication as before. So it’s known that the recent problems

clearly caused by defect of AVR on genset#1. If we see back to first event, it is noted that

genset#1 had an experience fail to auto-online due to AVR fail to synchronize with

system voltage. So this problem actually had existed since the first event.

The sequence of this event is as follow:

After genset#1 known got a problem with the voltage, there is additional event as follow:

Genset#3 Trip

Reactive Power

NO

BLACKOUT

G#1 &2

covered the

load because

the load shed

system work

Genset#5

Trip LoE

Genset#1, 2, 3, 5 was run online Genset#4 was under repair at HB

Genset#1

Trip

Overvoltage

Genset#1

attempted to

be ONLINE

This two events almost simultaneously

Genset#3 Trip

Reactive Power

NO

BLACKOUT

G#1 &2

covered the

load because

the load shed

system work

Genset#5

Trip LoE

Genset#1, 2, 3, 5 was run online Genset#4 was under repair at HB

This two events almost simultenously

From the above five-why analysis, we can count several root causes of frequent power

loss in Mutiara Central as follows:

1. AVR in genset#2 and AVR genset#1 had a problem lead to unstable operation

due to system voltage variations.

2. Bad ventilation system of genset#4 absorbed contaminant to winding lead to

degradation of insulation caused short circuit fault

3. The fuel system of engine is not good enough as it is frequently had adjustments

4. The relay setting of LoE, Reactive power and underfrequency still too sensitive

and need to be revised. It is recommended also to see the other protection setting

to be reviewed

1st Why?

Genset#5

Trip LoE

Genset#3 Trip

Reactive Power

- Power Factor hunting / excitation

current varies due to voltage variations

- Improper relay setting?

2nd

Why?

AVR defect lead to unstable

response when system fault or got

disturbances

Found unstable response of

AVR in genset#1.

Root causes:

- AVR Component damage

- Improper tuning

3rd

Why?

4th

Why?

5th

Why?

Low impedance or minus

KVAR reach the relay pickup

setting value

6. Recommendations

1. Install filter in cooling air inlet of genset#4

2. Install RTD in genset#3 stator winding and activate the protection in all unit’s

stator winding to prevent winding damage.

3. Review the load shedding system whether it can response faster?

4. Review the setting of protection relay to find any weak point, refer to generator

data provided by Trakindo. The most critical to be reviewed are:

- Loss of Excitation impedance circle of genset#5 need to be narrowed, so less

sensitive than before. As found setting 60 ohm, recommended to reduce to be 54

ohm. Already done.

- Reactive Power Limit setting of genset#3. Refer to generator capability curve.

- Under frequency setting genset#4. Increase time delay to allow load shed working

earlier than relay command to trip.

5. Modify the fuel system design to get better fueling to sustain engine power.

6. Detail task of PM Service to cover fuel injection solenoid verification.

7. Training maintenance personnel on how to adjust AVR correctly.

8. Find out tester or simulator for Automatic voltage Regulator.

Appendix 1

Email References

1. Email form Pak Zainal regarding adjustment of AVR in genset#2

2. Email from Pak Zainal regarding problem AVR in genset#1

Appendix2

Relay Setting

The overall setting review is in progress.

The critical setting need to be revised ASAP:

- Setting Reactive Power at genset#3

- Setting Loss Excitation of genset#5

Figure1. Rubah setting alarm reaktif power genset#3

Rubah setting

menjadi “No relays”

Figure2. Setting Impedance Loss of Excitation

Berdasarkan data reaktansi sinkron yang telah didapatkan (Xd=1.5970 ohm; X’d =

0.1278 ohm; X”d = 0.0908 ohm), direkomendasikan setting impedansi LoE dirubah

seperti yang warna merah.

Figure3 Setting Loss of Excitation di GMR

Appendix2

Event Logger

Event Log Genset#3

Event Log Genset#4

Event log Genset#5

Example of Electrical Utility Problem Investigation using 5-why methode

Documents

Transcript of Example of Electrical Utility Problem Investigation using 5-why methode