www.inl.gov

Computer Based Procedures

Insights From Field Evaluations at Nuclear Power Plants

Johanna OxstrandIdaho National Laboratory

Procedure Professionals Association SymposiumSt. Petersburg, FL, June 23-25, 2015.

Computer-Based Procedures (CBPs)Research Objectives• Define design requirements• Streamline and distill information• Use dynamic presentation to:

– Increase efficiency – Improve the ease of use– Reduce opportunities for errors– Incorporate human performance

tools• We do NOT investigate how to

display a document on an electronic device Field Evaluations

Catawba

2014

Palo Verde 2014

Diablo Canyon

2015Vogtle2015

Development and Evaluation of PrototypesPalo

Verde 2012

Catawba 2012

INL2013

Palo Verde 2014

Qualitative Study (2012)Model of

Procedure UsageIdentification of Requirements

Why is research on CBPs needed?• Standardized design guidance

for CBP enables– Taking multiple sources of disparate

information and present it in a succinct and easily comprehensive manner

– Human error reduction and increased efficiency

– Maintaining field workers’ focus on task at hand

• Advanced technology provides the opportunity for– Automatic job aids (e.g.,

calculations) and place-keeping – Digital correct component verification– On demand just-in-time training and

info (e.g., photos, videos)– Real time plant configuration and

control status

Design Concepts• Dynamic Context Sensitivity• Simplified Step Logic

• Digital Correct Component Verification• Automatic Place-Keeping

Examples

Field Evaluations• Objectives:

– Verify that the prototype system can be used in the real world with a real work package.

– Evaluating new design concepts and functionality.

– Identify the barriers and challenges.– Evaluate usability of the system

based on real-world experience. • Method:

– Plant work package is loaded in the prototype system.

– The system is used to conduct tasks in the plant.

– Data collection.

Field evaluations vs Laboratory studies • Real world conditions• More complex tasks• Longer duration• More dependent on

field workers’ feedback

Field Evaluation Activities• Conducted

– Catawba• April – May, 2014.

– Palo Verde• September 2014 – February, 2015

• Ongoing – Diablo Canyon

• April, 2015– Vogtle 1 or 2

• June, 2015

• Future studies– Would you like to host a

study?

Duke Energy’s Catawba Nuclear Station• Functional test of back-

up air compressor–Weekly test of one of

three compressors• CBP functionality– Context sensitivity– Continuous actions–Multiple branching– Additional info in steps– Digital CCV– Nexus 4

Results from Catawba • Conditional logic increased

efficiency and simplified task flow– Streamlined sequence of

complex steps– Fast transfer between

procedure sections• Most appreciated

functionality:– Photos of components– Documents are easily

accessible– Automatic placekeeping– Digital CCV

Feedback

“I wish I had this when I was new! Instead I spent a lot of time trying to locate components in the plant.”

“This would be very useful during outage or anytime you’re schedule to do a job you only do once in a while.”

APS’ Palo Verde Nuclear Generating Station• HVAC preventive

maintenance –Weekly readings on all

three units’ HVAC system

• CBP functionality– Previous logs– Validate input (in range)– Trending–Multiple users –Multiple days– Nexus 7

Results from Palo Verde• Autonomy in task flow

is hard to predict and design– Computerized support

and guidance?– “Skill of the Craft”?

• Low response and load times are essential for success

Table vs. CBWO

Table vs. CBWO

Table vs. CBWO

PG&E: Diablo Canyon Power Plant (DCPP)• Initial plant visit: February, 2015• Launch of study: April, 2015

Field evaluation study (using two separate tasks):• Swapping Auxiliary Salt Water Pumps

– Shared between control room operator and operators in the field

• Swapping Ventilation Supply and Exhaust Fan Sets– Field operators

• Nexus 10 and Nexus 7

Preliminary Feedback - Most appreciated functionality• Available information – Pictures of valves– OVID drawings with

highlighted flow path• Automatic placekeeping – N/A-ing of steps

Preliminary Feedback - Future functionality• Valve locations• Stop and restart in the

middle• Automatic update of

plant logs• Messaging function

(e.g. to send updates to Work Control)

• Support left handed operation of the devices

Southern Nuclear Company: Plant Vogtle 1 & 2• Initial plant visit: March, 2015• Launch of study: June, 2015

Field Evaluation Study• Reoccurring Battery and Charger

Inspection and Maintenance • Task shared between

Maintenance Technicians and Operations

• Automatic population of data sheets

• Printable copy for archiving• Nexus 10 and Nexus 7

New Layouts – Quick Evaluation• Do you have 5 minutes to spare?• Help us find out what you like and don’t like with

our new layout options!

? ??

We look forward to future collaboration endeavors! Johanna.Oxstrand@inl.gov, Katya.LeBlanc@inl.gov, and Aaron.Bly@inl.gov

SCREENSHOTS AND DESCRIPTIONS

INL Prototype System as of October 2014

Login Screen• Log in by either typing

name or scanning barcode on badge.

• Option to log out or switch users.

• Selection of work order or procedure.

• Option to select ongoing work order or create a new instance.

Work Order Summary• Provides the same

information as the coversheet on the paper-based version.

• Barcode is generated based on instance information inserted when logging in to the system.

• The navigation menu will appear when swiping right or using menu option (see red circle).

Automatic Placekeeping• The step to take action on is

marked with the blue boarder. Past and future steps can be reviewed but no action can be taken.

• In this example, the technician decides in what order to take the readings, i.e., which chiller to go to in what order.

• The technician can exit the readings before they are complete and come back to complete them later.

• Conducted steps have a blue checkmark and displays the taken action, decision, or reading.

Correct Component Verification (CCV)• Option to conduct Digital CCV

by scanning a barcode associated with the component.

• The system will notify the user if incorrect component barcode was scanned, i.e., if the verification was conducted on wrong component.

• When the user has located the component he/she will have the option to either re-scan the barcode or conduct a manual component verification.

Conditional Statements• Conditional statements are

presented as binary questions.• Based on the answer, the

system will take the user down the relevant path through the instructions.

• The system will mark non-relevant steps as Non Applicable (N/A’d).

• Second screenshot shows the system updating the instruction based on the input that the chiller is not running. Steps associated with a running chiller will be N/A’d.

Taking Readings• Readings from previous logs are

automatically populated in the instruction (e.g., Refrigerant 124A’s Last Log Level)

• To insert a reading the user clicks on the text field and the keyboard will appear.

• To edit previous entered values, the user clicks on the value.

• In this example the readings can be taken in any order, hence there is no automatic placekeeping to guide the user to the next relevant step.

• Conducted steps are marked with a checkmark.

Calculations• Calculations are automatically

conducted by the system based on the readings.

• The user has the option to view the calculation for review purposes.

• The user has to accept the result or override it. If the calculation is overridden, the user has the option to insert a comment to document the decision.

Trending• Trends based on previous logged

readings are available to the user.

Other Functions• The system informs the user of

units to be used.• There is an option to provide

comments at each step (see red circle). There is also a general comment section in the work order. All comments (incl. the ones made on individual steps) are saved in the general comments.

• If abnormal readings are encountered, the user is required to document actions taken.

Locate place-keeping function on page Visibility of place-keeping function

Visual search: Number of distracters and saliency of those distracters

• Identify actions to be taken• Compare actions to mental model

• Understanding of constrained language• Complexity of step logic• Accuracy of mental model

• Working Memory• Long-term Memory

Compare expected conditions to actual conditions

• Accuracy of mental model that dictates expected conditions• Detection of cues that indicate departure from expected conditions

• Salience of cues that indicate departure of expected conditions• Long-term Memory

Compare location, label, and equipment description in procedure to those on the actual equipment

• Ability to correctly read both the procedure and the equipment information• Ability to integrate the values/text read and compare them accurately

• Text comprehension• Working Memory

Compare expected conditions to actual conditions

• Accuracy of mental model that dictates expected conditions• Detection of cues that indicate departure from expected conditions

• Long-term Memory• Salience of cues that indicate departure of expected conditions

Check for indications that there are more parts to the step (e.g., was the current action a sub-step?)

• Existence of indications that signal the step is not complete• Visibility of indications that step is not complete

• Prospective Memory• Salience of cues

Identification of cues that signal dependency (i.e., warning)

• Existence of cue that signals dependency• Visibility of cue

• Prospective Memory• Salience of cues

Technique Cognitive factors that determine task success or failure

Conditions that must be satisfied to ensure task success

Start

Read procedure step

Do I understand step?

Do other steps depend on this step?

Develop action plan/strategy

Locate component

Are the initial conditions as expected?

Predict expected system response

Execute step

Was step executed correctly?

Was expected response achieved?

Yes

Next step

Yes

Evaluate situationNo

Call supervisior

No

No

No

Yes

Yes

Yes

No

Yes

Right location, label, and name?

Am I on the right step?No

Is the step complete?

Yes

No Yes

No

Yes