▪ ▪ CLARITY ▪ ASSURANCE ▪ RESULTS MIDWEST RELIABILITY ORGANIZATION Improving RELIABILITY and...

CLARITY ▪▪ ASSURANCE ▪▪ RESULTS

MIDWESTRELIABILITYORGANIZATION

Improving RELIABILITY and mitigating RISKS to the Bulk Power

System

Thomas P. TierneyDirector of Compliance, MRO

SPP Compliance ForumMay 23, 2013

Reliability Assurance Initiative

CLARITY ▪▪ ASSURANCE ▪▪ RESULTSMay 23, 2013 2

Common MissionCommon Mission

Improve the Reliability

of the Bulk Power System


Improvement Is the GoalImprovement Is the Goal

We have very reliable systems within MRO/SPP, but we can still improve by identifying problems and fixing them – no weak links

There is always opportunity for improvement within the design criteria of an interconnected system

May 23, 2013 3


Demystifying Internal Controls

No, Really… What Is an Internal Control?


Nothing NewNothing New

Registered Entities have been managing reliability for decades – they have management practices (i.e. controls) around reliability

Existing practices have been translated into the Reliability Standards and documented – “operationalizing compliance”

Don’t overthink “internal controls”

May 23, 2013 5


Risk• Possibility that something undesirable will happen• Measured as a combination of likelihood and impact

Control/Control Activity• Policy, procedure, checklist, etc. designed to minimize

the opportunity for a risk to be realized

Internal Control• Control activity performed internally, not by a third

party• Management practices that include control activities

performed internally (“self monitoring”)

May 23, 2013 6

DefinitionsDefinitions


Inherent Risk• Risks “built-in” to a given entity, based on geography, what

facilities it operates, “interconnectedness,” etc.• Reliability Standards are designed to mitigate inherent risk in a

broad sense

Control Risk• Risk that management practices or control activities are not

achieving their reliability or compliance objectives• Detection Risk

• Risk that possible violations are going unnoticed• Residual Risk

• Risk that remains after application of a control and other mitigating factors

• Difficult and expensive to eliminate 100% of risk – we must live with some risk

May 23, 2013 7

Types of RiskTypes of Risk


Preventive• Controls designed to stop something from occurring

Detective• Controls designed to identify when a possible violation

has occurred and facilitate timely remediation• Also known as “Monitoring” controls

May 23, 2013 8

Types of ControlsTypes of Controls


Multiple, complementary controls that work together to reduce risk (“Defense in depth”)• Primary• Secondary• Tertiary

Secondary and Tertiary controls serve as a “safety net” in case the Primary control does not function as expected

Each subsequent tier of controls further reduces residual risk

May 23, 2013 9

Control HierarchiesControl Hierarchies


Protection System Maintenance and Testing• Relay technicians complete work orders according to a

pre-defined checklist to prevent steps being skipped or performed incorrectly

• Supervisors review and approve completed work orders to verify technicians’ proper use of the checklist

• A sample of work orders is reviewed by Internal Audit to verify accuracy and completeness

May 23, 2013 10

ExamplesExamples


ProgramDocuments

(Procedures)Standard

Work Order

Supervisory Review

Management Oversight

Checklist followed and completed, exceptions noted, follow-up notes signed

Review for completeness and accuracy, follow-up actions closed or scheduled to be completed, signed

Periodic sampling of work orders to determine program is being completed and properly reviewed

Procedure/Process Control Control Activity Control Type

Primary Control

Secondary Control

Tertiary Control

ExamplesExamples


Training• Management establishes training objectives and

reviews training materials to confirm objectives are met• Individuals are tested after completion of training to

ensure effectiveness of delivery• Supervisors conduct performance observations to

verify past training has been effective and to identify additional training needs

May 23, 2013 12

ExamplesExamples


ProgramDocuments

(Procedures)Training

Objectives

Training Evaluation

Performance Observations

Management establishes training objectives and reviews training materials to confirm objectives are met

Individuals are tested after completion of training to ensure effectiveness of delivery

Supervisors conduct performance observations to verify past training has been effective and to identify additional training needs


Primary Control

Secondary Control

Tertiary Control

ExamplesExamples


Cybersecurity• Systems are configured to require passwords to

prevent unauthorized access• All changes to systems are reviewed, approved, and

tested to ensure that unauthorized changes do not occur

• Periodic reviews are conducted to ensure that password controls adhere to corporate security policies

May 23, 2013 14

ExamplesExamples


Security Policies

Password Controls

ConfigurationManagement

Security Assessments

Systems are configured to require passwords to prevent unauthorized access

All changes to systems are reviewed, approved, and tested to ensure that unauthorized changes do not occur

Periodic reviews are conducted to ensure that password controls adhere to corporate security policies


Primary Control

Secondary Control

Tertiary Control

ExamplesExamples

Configuration Management Procedures


Reliability Assurance Initiative

Focusing on Risk


“One size fits all” compliance model• NERC Actively Monitored Standards do not change based

on regional differences, entity size, etc.• No consideration of management practices (i.e. controls)

around reliability standardsZero-defect approach to enforcement is burdensome• Every violation requires a regulatory filing regardless of

severity• Self-reports require significant effort

Administrative Citation Process (ACP) & Find, Fix, Track (FFT) are not sufficient• Expediting enforcement won’t solve the problem

May 23, 2013 17

Current StateCurrent State


Shape compliance monitoring and mitigation based on risk

Reserve enforcement for most significant risks

May 23, 2013 18

Key Elements of RAIKey Elements of RAI


Assessment of each entity’s inherent risk

Some factors influencing assessment• Facilities• Special Protection Systems• IROLs• Geographic location• Functions performed• Connectivity (physical and cyber)• EMS/SCADA system• Compliance history

May 23, 2013 19

Scoping of WorkScoping of Work


What does a risk assessment look like?• Not a letter grade or single rating• Entities will not be compared and ranked

Assessment will look more like a matrix• Certain families of standards may be higher risk for

one entity, less risky for another

May 23, 2013 20



Internal controls established by each entity must be identifiedEvaluation of select controls to determine effectiveness• Design – Is the control, as documented, adequate to

address the risk?• Operational – Is the control implemented as designed?

Effective controls reduce residual risk to an acceptable levelMRO staff can rely on effective controls• Regulatory scope can be adjusted – less auditing and

testing (or none) where strong controls exist

May 23, 2013 21



Risk assessments and internal controls will be leveraged across all compliance monitoring activitiesInternal emphasis should shift over time toward maintaining effective controls around Reliability Standards• Continue to identify and correct issues in a timely

fashion• Focus on reliable operations first• Compliance should be a natural outcome of strong

operations

May 23, 2013 22



“Compliance Exceptions” represent lower risk violations• Do not represent significant risk to the BES• Identified by an entity itself or by regional staff• Initially tracked at the regional level• No enforcement proceedings, no penalties

Mitigation will always be important• What was done to address the problem itself?• What is being done to prevent recurrence?

May 23, 2013 23

Compliance ExceptionsCompliance Exceptions


Enforcement will focus on most significant or high-risk issues• Violation poses significant risk to reliable operation of

the BES, e.g. cause or contributing factor in a cascading event

• Multiple smaller issues may aggregate into a bigger problem or are indicative of a poor control environment

• Willful misconduct

May 23, 2013 24

Compliance ExceptionsCompliance Exceptions


Compliance audit• Tools being developed with input from industry, the

Regions, and NERC• Currently developing risk assessment• Internal controls evaluation to occur during June & July• Scope will reflect risk and presence of effective

controls• Audit completion in Q4 of 2013

May 23, 2013 25

MRO PilotsMRO Pilots


Self-certification• Transition from blanket, “check the box” approach to

narrowly focused self-certifications• Scope limited to FAC-008-3 R6 based on problems

identified on recent audits• Focus on self-assessment process and on controls to

identify and correct problems

May 23, 2013 26

MRO PilotsMRO Pilots


Contact InformationContact Information

Thomas P. Tierney, Director of Compliance

Midwest Reliability Organization

[email protected]

(651) 855-1745

May 23, 2013 27


Questions?Questions?

May 23, 2013 28

▪ ▪ CLARITY ▪ ASSURANCE ▪ RESULTS MIDWEST RELIABILITY ORGANIZATION Improving RELIABILITY and...

Documents

Transcript of ▪ ▪ CLARITY ▪ ASSURANCE ▪ RESULTS MIDWEST RELIABILITY ORGANIZATION Improving RELIABILITY and...