Canadian Electrical Association

Engineering and Operating DivisionPower System Planning and Operating Section

April 1996Montral, Canada

Fundamental Reliability Considerations in theDesign, Manufacturing and Application of

Multifunction Digital Relays for Generator Protection

byCharles J. Mozina and Dr. Murty V.V.S. Yalla

Beckwith Electric Co., Inc.6190-118th Avenue North

Largo, FL 34643-3724 USA

SUMMARYA major concern in the application of multifunction digital relays for the protection of generators is that almost

all the electrical protection, including both primary and the backup relay functions, are embodied in one digitalpackage. The failure of that package results in the loss of virtually all generator electrical protection. This paperdiscusses how multifunction relays can be designed and manufactured to minimize such failures. It describes therole of self-diagnostics in determining the health of the relay. Most importantly, it proposes cost-effective appli-cation strategies which can be employed to survive an in-service multifunction relay failure without loss ofgenerator protection.

Keywords: multifunction relay, self-diagnostics, mean-time-between-failures (MTBF), redundant protectionsystems.

INTRODUCTIONAs all relay engineers are aware, protective relay technology over the past twenty-five years or more has

evolved from single-function electromechanical relays to static relays and finally to digital relays. The firstdigital relays were single-function units. However, as microprocessors became more powerful, designers soonsaw the economic advantage of designing multifunction relays. In these relays, virtually all protective functionsfor a specific protective zone are incorporated into a single hardware platform. Figure 1 illustrates the number ofprotective functions which can be installed on a single hardware platform for generator protection. A failure ofthe hardware platform will typically disable all protective functions within a protective zone. Therefore an impor-tant issue in the application of multifunction digital relaying is how to handle having all the eggs in one basket.The installation of both independent primary and backup protection is one of the most fundamental concepts ofprotective relaying.

Utility System

52Unit

High-ImpedanceGrounding

3

IN

52Gen

SingleHardwarePlatform

M-3430

3

50BF-N

59N

27

59 24

27TN

81 27

27

5050BF

27

46

87

21 3260FL 40

SingleHardwarePlatform

Figure 1 Protective Functions

In recent years, some manufacturers have argued that with self-diagnostics (the ability of the relay to checkitself), a relay failure would be immediately known and the protected piece of equipment could be removed fromservice until the relay was replaced or repaired. Most users have found this philosophy unacceptable. This isespecially true in relation to generator protection. Even with a mean-time-between-failure rate of 70 years ormore (based on in-service operating experience with digital relays), the consequences of removing a major gen-erator from service due to a single relay failure are unacceptable to most users. The loss of a major generatorimmediately increases the cost of generation for a utility for the time the machine is out of service. The utilitycompensates for this lost generation by either running less efficient generation in-house or purchasing moreexpensive power off-system. Even the loss of a moderately-sized (200 MW) generator can cost a utility and itscustomers $100,000 per day in added fuel or purchased power costs. In addition to the economic consequences,many relay engineers fear the failure of a digital relay could occur concurrently with a protection event when therelay is necessary to protect the generator.

This paper explores these reliability issues from the viewpoint of both a manufacturer as well as a user. Itdiscusses one manufacturers experience with in-service failures, as well as the design, manufacturing and testingto reduce or eliminate such failures. The role that self-diagnostics can play in the development of applicationstrategies to survive a single relay failure is also presented.

DESIGN CONSIDERATIONS TO MINIMIZE FAILURESIn the 1960s, solid-state electronic protective relays using discrete components were developed. These relays

used many discrete components and associated interconnections and therefore were not as reliable as equivalentelectromechanical relays. Continued developments in the semiconductor industry led to the introduction of inte-grated circuits which combined complex electronic circuits into single chips. This use of integrated circuitsreduced the number of components and enhanced the reliability of the solid-state protective relays.

In the 1980s, the advent of microprocessors and high-speed digital signal processors brought a new genera-tion of relay designs. These digital relay designs contained less hardware but required considerable softwaredevelopment. Using self-diagnostics, these relays can detect most relay failures and alert maintenance personnelusing alarm indication.

Hardware ReliabilityThe block diagram in Figure 2 shows the hardware architecture of a digital multifunction relay used for

generator protection. The voltage inputs of the relay are scaled down from the nominal 120/69 V to a low leveldetermined by the ADC (analog-to-digital converter) input range. The current inputs to the relay are scaled downfrom nominal 5/1 A and are converted to equivalent voltages. These scaled signals are filtered using a low-passfilter to prevent aliasing of the high-frequency components into the fundamental frequency component. Thefiltered signals are multiplexed using an analog multiplexer and amplified, if needed, using a programmable gainamplifier. The multiplexed analog signal is sampled and converted to digital data using the ADC. The relay usesa dual-processor architecture wherein the digital signal processor executes complex algorithm calculations andthe host processor performs all other tasks. Communication between the two processors is provided by the dual-ported memory. Flash memory is used for storing the program and RAM (random-access memory) is used fortemporary storage of variables, target information and oscillography. Contact inputs and outputs, user interface(keyboard and liquid crystal display) and the serial communication ports (RS-232 and RS-485) are interfaced tothe host processor.

The digital signal processor executes a variety of signal-processing algorithms to estimate several parametersof the digitized voltage and current signals and transfers them to dual-ported memory. The host processor re-ceives these parameters from the dual-ported memory and performs relay logic and other timing functions togenerate appropriate trip or alarm output signals. The host processor, running under a multitasking operatingsystem, also performs several other tasks including: communications, setpoint updates, target updates, and userinterface.

VTs & CTs

2K byteDual-Ported

RAM

Digital SignalProcessor

(DSP)TMS 320C52

32K X 16RAM

14-bitAnalog-to-Digital

Converter(ADC)

i

va

b

c

n

a

b

c

A

B

C

N

ProgrammableGain Amplifier

Address/Data Bus

256K byte Flash-Programmable

ROM

Host Processor10 MHz Zilog 64181

MMIModule

(Optional)TargetModule

(Optional)RS232 and RS485Communication

ports

Analog Multiplexer

Anti-Aliasing Low-Pass Filters (LPF)

IRIG-BTime Code

inputRelay

Outputs

2-Line by 24-CharacterLiquid Crystal Display

128K byteRAM

512 byteEEPROM

8K byte RAM,Clock with

batterybackup

Power Supply

i

i

i

v

v

v

i

i

i

(Optional)Power Supply

ContactInputs

MUX

Figure 2 Block Diagram

Possible hardware design problems can be identified by performing a worst-case analysis during the designstage. Such problems, which include voltage and current stresses, extreme temperatures, power dissipation andtiming requirements, can then be corrected. This can greatly reduce hardware failures in the field. Failures canalso be reduced by using extended-temperature components, conservatively derating the components to lowerstress levels, and utilizing components from established and reliable vendors.

Providing backup for critical components can also enhance the reliability of the relay. The digital multifunc-tion relay described above uses redundant power supplies. Both power supplies are continuously running in a hotstandby configuration and should one supply fail, the other will continue to provide required uninterrupted powerto the relay. The relay also sends an alarm indication about the power supply failure to alert maintenance person-nel.

The analog signal inputs (voltage and current), contact status inputs, contact outputs, power inputs, and com-munication circuits must be conditioned and protected to withstand the harsh electrical and environmental condi-tions of the substation and power plant.

The design of relay input, output and power supply circuits must incorporate filtering to reduce EMI (electro-magnetic interference). The primary method of reducing unwanted induced ac voltage is to bypass these voltagesto ground with capacitors. Other components, such as varistors, chokes and ferrite beads, are also applied tosuppress surge voltages and EMI.

Software ReliabilityThe reliability of software in the digital relay is critical to the overall reliability of the product. The majority of

the software problems in digital relays can be attributed to design and implementation errors. More than half ofall the errors occur long before the first line of code is written, i.e., during requirements analysis and top-leveldesign. Most of these errors are caused by poorly-defined requirements but very few errors are detected whenthey occur.

A majority of the remaining errors occur during the detailed design phase of the development, mainly due topoor translation of the users requirements into the programs and data. These problems can be minimized bycarefully planning and designing before coding starts, resulting in a more reliable product. A software qualityassurance plan must be carried out throughout the product development program. Test plans, documentation,detailed software validation and audit programs can greatly reduce software errors. Product failures that are notdetected early in the design stage can be very costly to fix when they happen in the field.

Software verification and testing of multifunction relays offer unique challenges to relay manufacturers. Test-ing should be divided into several categories and should be conducted at various design phases. The following aresome of the key tests conducted on digital multifunction relays for generator protection.

1. Relay algorithm simulation testing

2. Static functional testing

3. Dynamic functional testing

4. Environmental and hardware-related tests

5. Beta-site installation and testing

SELF-DIAGNOSTICSSelf-diagnostics is one of the most important features of digital relays; it was not available in either electrome-

chanical or static relay designs. The ability to detect and correct a failure before the protection system has tooperate contrasts to traditional protection systems where a relay failure remains undetected until it fails to operatecorrectly during an event or until the next maintenance test. The quality of electronic components available todayis excellent; however, failure of these components can still occur. Digital relays can be designed to detect most ofthese failures. The following are some of the most important self-diagnostic functions implemented on digitalmultifunction relays.

1. Data acquisition system testing

Power supply voltages and ground are connected to the analog input channels of the multiplexer and checkedagainst warning and failure thresholds. This also verifies the analog data acquisition system including: multi-plexer, programmable gain amplifier, and ADC. The ADCs conversion time is also checked to see if it is withinthe specification.

2. Memory testing

The flash ROM contents are checked by calculating the checksum and comparing it to the pre-computed andstored checksum. The checksum is calculated as the modulo-256 sum of all the bytes. The RAM is tested bywriting and reading a test pattern.

3. Setpoint testing

Setpoints are stored in the serial EEPROM and a copy of these setpoints is also stored in the RAM forexecuting relay logic. Whenever any setpoint is changed, the checksum of the setpoints is calculated from thecontents of the EEPROM. This checksum is then compared with the calculated checksum of the setpoints storedin the RAM every time a setpoint task is executed.

4. Watchdog timer

The relay hardware design includes a watchdog timer reset circuit to take the processor through an orderlyreset should the program get lost due to hardware/software glitches.

MANUFACTURING METHODS TO MINIMIZE FAILURESMajor efforts are made at every step of manufacturing to eliminate failures. At the component level, attempts

are made to weed out faulty components. Some components are 100% tested, while others are sample-tested.The decision as to which type of testing to do is based on experience with the failures of that particular compo-nent. Beckwith Electric selects and uses components that are industrial-grade or better. Component selection andapproval comes after extensive testing by the Quality Assurance Department.

Once components are tested, the circuit boards are built. These are then tested to determine if there are anycomponent failures, poor solder connections, improperly installed components or open connections. Once thistest is passed, the relay is assembled and again tested. Each relay is heat-cycled for 100 hours. It is programmedwith factory settings and comprehensively tested with currents and voltages using a computer-driven, three-phasetest set. Each relay is also subjected to industry surge tests such as the SWC and Fast Transient Tests outlined inANSI/IEEE C37.90. To minimize the chances of static discharge failure due to handling of the components andcircuit boards, the floor of the plant has been covered with an anti-static coating in addition to using groundedhandstraps as a standard part of the manufacturing process.

Beckwith Electric has been building multifunction digital generator relays since 1989 and has over 1000 unitsof our first generation of digital relays in-service around the world. Failure statistics are carefully kept to attemptto determine if there is any pattern or specific components that fail. To date, we have had a total of 20 in-servicefailures in over 13.6 million operating hours; our mean-time-between-failures (MTBF) rate is slightly over 74years. We have not detected any pattern to these failures. All failures have been detected by self-diagnosticsoperating as designed (the relay being automatically removed from service without tripping the generator). Eachyear, as we put more relays in-service, our MTBF rate also increases.

Another way to look at these statistics is that if you have ten relays in-service for twenty years, you couldexpect roughly one failure. This holds true, however, only if past performance is reflective of future performance.Is there an aging factor? Does the number of years that the relay has been in service decrease the performance ofthe relay? Most industry experts say no, but this has not been proven. Based on the performance to date, one cansay that this technology has an excellent reliability record.

LEVEL OF REDUNDANCYGiven the performance level of digital generator protection, what is the appropriate level of redundancy? On

larger generators protected by digital relays, the use of fully redundant systems is justified. Such a scheme isshown in Figure 3.

This system has been adopted by a number of users, including two major manufacturers of large (100 to 150MW) gas turbines. This level of redundancy is sufficient to allow the generator to remain in service if one relayshould fail. If a major generator is forced off-line due to a relay failure, the utility/generator owner will have toeither generate from less efficient machines or buy more expensive power off-system. Either action will result inhigher production costs of over $100,000 a day for the loss of a moderately-sized utility generator. Given thesecosts, the addition of a second relay is certainly prudent even with MTBF rates that are 74 years or better. Thesimultaneous failure of both relays is extremely rare. Even with two digital relays, the installation cost is gener-ally less than half the cost of discrete static or electromechanical protection costs, due to panel space and wiringcost savings. A typical panel comparison is shown in Figure 4.

59N

M-3420

M-3420

3

3

Utility System

52Unit

High-ImpedanceGrounding

3

IN

52Gen

M-3430

M-3420

M-3430

3

50BF-N

AVR 1

59N

27

59 24

8127

50

50

27

27

51V32 60FL4046

24

50BF

87

59

27TN

81 27

27

27

AVR 2

51N

50

87

50BF

46324060FL 21

Figure 3 Dual-Relay Protection Approach for Major Generators

40 40

AB

21

BC

21

CA

21

32

3

Directional PowerRelay Function

ELEMENT#2

24

B

59

C

5959I59

A

Third HarmonicNeutral Undervoltage

Relay Function

27TN

GroundDifferential

Relay Function

87GD

81O/U

Over/UnderFrequency

Relay Functions

59N RMS OvervoltageNeutral

Relay Function

ELEMENT#1

50BF50N

BreakerFailure/FlashoverRelay Functions 3 3

60FL

VTFuse-LossDetection

46NegativeSequence

Relay Functions

27/59

InadvertentGenerator

Relay Function

Volts per HertzRelay Function

Phase DistanceRelay Functions

Phase VoltageRelay Functions

Loss of FieldRelay Functions

Figure 4 Panel Space Savings

The design of the self-diagnostics in the multifunction relay is such that if a failure is detected, the relay willautomatically take itself out of service and close its alarm output contact. The self-diagnostics is designed toremove the relay from service without tripping the generator. To date, this design has been 100% successful withno in-service failures resulting in the tripping of a generator. Also, all in-service failures were successfullydetected by the self-diagnostics.

Is dual protection necessary on all sizes of generators? The answer is clearly no. If a relay fails, the genera-tor must be removed from service, by either manual or automatic tripping methods. If the cost of taking agenerator off-line for a few days to replace a relay is not significant, then a single relay is adequate. The genera-tor owner must balance the cost of an additional relay against the probability of a relay failure over the life of theinstallation. With a MTBF rate of 74 years or better, smaller generators can be protected with a single relay. Dualprotection is justified when the cost to the generator owner for the loss of the generator is significant.

Some people have suggested that important generators be protected using two-out-of-three logic. This type oflogic has been used at nuclear plants for some types of protection such as second-level voltage separation. It hasnot been used to protect generators, even at nuclear plants. Figure 5 illustrates this logic.

The use of the third relay adds security against false tripping by requiring a second independent relay toconfirm that tripping is required. Thus, if a relay fails and gives a erroneous trip signal, no tripping will takeplace because a second relay output is required. In our view, two-out-of-three logic is an unnecessary complica-tion because of the self-diagnostics designed to remove the relay from service without tripping the generator.With Beckwith Electrics first generation of digital relays, field experience to date has been 100% successful withno in-service failures resulting in generator tripping.

VT

CT

Relay1

Relay2

CT

Relay3

CT

86G GeneratorLockout Relay

(+)

(-)

Relay 1

Relay 2

Relay 2

Relay 3

Relay 3

Relay 1

Figure 5 Two-Out-Of-Three Logic

IMPACT OF SELF-DIAGNOSTICS ON PERIODIC MAINTENANCEOne of the major benefits of relay self-diagnostics is its impact on periodic maintenance. With conventional

electromechanical and solid-state electronic relays, the user has to verify that the relay is operating properly byperiodically injecting currents and voltages. Most utilities do this every two to three years. Another source offrequent failure is the external wiring connections between primary relays to provide the logic required fortripping. Generator protective relays are frequently supervised by VT potential failure (60) logic, generator cir-cuit breaker position and generator terminal voltage. This logic frequently requires the use of numerous auxiliaryrelays which reduces the overall system reliability. In todays modern multifunctional digital relay, this type oflogic is programmed into the relay. Once programmed, it is checked by the same self-diagnostics as is theprimary relay logic itself.

What type of periodic maintenance is meaningful for digital relays? We believe the user should periodicallycheck the inputs to the relay. Extensive input metering information can be accessed either via computer or locallyby the man-machine interface. This information indicates the relay is receiving proper input data. An example ofsuch a metering computer screen is shown in Figure 6.

120.0 120.0 120.0 0.0 120.0 0.0 0.0 3.00

5.000 5.000 5.000 0.010 5.000 0.000 0.000

5.000 5.000 5.000 0.00 0.00 0.00

100.060.001.00 LAG0.00001.0000

24.00 0.00 24.00 0.00 24.00 0.00 24.00 0.00

Figure 6 Computer Metering Screen

The user should also periodically activate the digital relay trip output contacts to verify that they are workingand are wired to perform the desired external tripping and alarming. A convenient means should be provided tosequentially activate each of the output relays to facilitate this type of trip testing. Both types of input and outputfunctional tests described above should be done on a periodic basis. The need to do costly and time-consumingcurrent and voltage injection testing has been significantly reduced by self-diagnostics. Many utilities have ex-tended the period for this type of testing from two or three years to ten years or longer. This is a significantmaintenance cost savings provided by digital technology.

CONCLUSIONSThis paper describes how the design and manufacturing methods are used to reduce in-service failures of

digital multifunction generator relays by a major manufacturer. It presents the resulting failure statistics based onover 13 million hours of in-service experience. Even with high reliability levels, the use of redundant protectionis recommended for major generators where digital multifunction relaying is the sole source of protection. Todetermine the generator size at which a second redundant relay is justified, measure the cost of the generator lossfor the time it takes to install and commission a new relay.

The user must balance the costs of an additional relay against the probability of a relay failure over the life ofthe installation. Two-out-of-three logic is an unnecessary complication because self-diagnostics is designed toremove the failed relay from service without tripping. Field experience to date has been 100% successful with noin-service failures resulting in generator tripping. The maintenance impact of self-diagnostics results in a majorsavings allowing the user to substantially extend the current- and voltage-injection testing period. Functionaltesting of the relay inputs and outputs, however, is recommended on a more frequent basis.

REFERENCES[1] A Digital Multifunction Relay for Intertie and Generator Protection, Murty V.V.S. Yalla and Donald L.

Hornak, Canadian Electrical Association, March 1992.[2] A Digital Multifunction Protective Relay, Murty V.V.S. Yalla, IEEE Transactions on Power Delivery,

Vol. 7 No. 1, January 1992, pp. 193-201.[3] Upgrading Generator Protection Using Digital Technology, Charles J. Mozina, Canadian Electrical Asso-

ciation, March 1995.

BIOGRAPHIESChuck Mozina is currently Manager of Application Engineering for Protection and Protection Systems for

Beckwith Electric Co. He is responsible for the application of Beckwith products and systems used in generatorprotection and intertie protection, synchronizing and bus transfer schemes.

Chuck is an active member of the IEEE Power System Relay Committee and is the past chairman of theRotating Machinery Subcommittee. He is the U.S. representative to the CIGRE Study Committee 34 on SystemProtection and chairs a CIGRE working group on generator protection. He also chaired the IEEE task force whichproduced the tutorial The Protection of Synchronous Generators.

Chuck has a bachelor of science in electrical engineering from Purdue University and has authored a numberof papers and magazine articles on protective relaying. He has over 25 years of experience as a protectionengineer at Centerior Energy, a major investor-owned utility in Cleveland, Ohio. He is also a former instructor inthe Graduate School of Electrical Engineering at Cleveland State University.

Dr. Murty V. V. S. Yalla is currently Vice-President of Research and Development Engineering for BeckwithElectric Co. where he is responsible for the development of new products in the areas of digital control andprotection of power apparatus, and the design enhancement and engineering support of current products. He hadpreviously served as Beckwith Electrics director of research and development, staff engineer and senior engineer.

Dr. Yalla is a senior member of IEEE and is active in the Power System Relaying Committee. He has pub-lished several research papers on digital protection in various international journals and is the co-author of threepatents. Dr. Yallas degrees, all in electrical engineering, include: a bachelor of science degree from JawaharlalNehru Technological University, Kakinada, India; a master of science degree from the Indian Institute of Tech-nology, Kanpur, India; and a doctorate from the University of New Brunswick, Canada.

Prior to joining Beckwith Electric in 1989, Dr. Yalla taught and conducted research in the digital protection ofpower apparatus at Memorial University of Newfoundland, Canada.