Apply Risk Management to Computerized and Automated Systems

Apply Risk Management for Computerized and Automated Systems

IVTIVT11th Annual Change Control & 3rd Annual Risk Management

January, 2013

Presented By:

1

AgendaAgenda

I Terms & Definitions - Q9 Quality RiskI. Terms & Definitions - Q9 Quality Risk Management

II. GxP AssessmentIII. Risk Identification and PrioritizationIV. Risk Based Test PlanninggV. Periodic Reviews of RisksVI. Interactive Exercise

2

AgendaAgenda

Terms & Definitions - Q9 Quality Risk ManagementICH Q9 “Quality Risk Management”GAMP 5GxP Assessment

ProcedureForm21 CFR Part 11 relevance

Risk Identification and PrioritizationSeverityProbabilityRisk ClassDetectabilityRisk Priority

Risk based Test PlanningExamples

Periodic Reviews of RisksMaintaining appropriate risk levels.

Interactive ExerciseUsing a real life example, participants study Life Cycle Risk Management

3

Typical Risk Assessment Points throughoutTypical Risk Assessment Points throughout System’s Life Cycle

4

Terms & Definitions - Q9 Quality Risk ManagementTerms & Definitions Q9 Quality Risk Management

Risk Identification – What can go wrong?Risk Identification What can go wrong?Risk Evaluation – Severity, Occurrence, DelectabilityRisk Analysis – Quantitative (1 -5)Risk Analysis – Quantitative (1 -5)

Qualitative (High – Low)

Risk Control – Reduction AcceptanceRisk Control Reduction, AcceptanceRisk Communication/Review

5

GAMP ApproachGAMP Approach

Understand the processUnderstand the processUnderstand the product and dataQuality Management SystemQuality Management SystemScalable Life Science ActivitiesScience Based Quality Risk ManagementScience Based Quality Risk ManagementSupplier Involvement

6

GxP AssessmentGxP Assessment

GxP -- The collective requirements for processes, personnel, materials q p , p ,and equipment used in the manufacture and distribution of foods, drugs and medical devices as defined in 21 CFR for Good Manufacturing Practices (cGxP), Good Clinical Practices (GCP), Good Laboratory Practices (GLP) and Good Distribution Practices (GDP) GxP may alsoPractices (GLP) and Good Distribution Practices (GDP). GxP may also include practices and procedures considered to be “industry standards”.

This procedure describes how computerized applications and systems are assessed for GxP – relevance and 21 CFR compliance.

8


GxP Assessment QuestionsDoes the application control or monitor machinery or instrumentation used in the manufacture of product? This includes critical support systems for steam, compressed air, water for injection, and clean room air.I th li ti d t d t l l t d t d tiIs the application used to document or calculate product, production process, or material quality information? This includes defect count, defect types, inspection results, and QC sample information.Is the application used to document or track which materials were used in ppmanufacture or testing of a product or in-process material?Is the application used to document or calculate the results for a procedure defined on a material specification?D th li ti h d l t k th lib ti i t hi t fDoes the application schedule or track the calibration or maintenance history of items used in product manufacture or testing?Does the application track or control the issuance of GxP-related documents? Examples: NLR issuance, procedure issuance.p p

9


GxP Assessment QuestionsDoes the application provide the original record of an activity required by GxPs? Examples: GxP training, complaint investigations, procedurally required quality trending reports.I th li ti d t t th t bilit f d t t i lIs the application used to support the acceptability of products, materials, or processes?Does the application support (store e-records, perform calculations) a system or process validation?pDoes the application support issuance or distribution of product labeling, marketing literature, directions for use, or other similar controlled product literature?El t i R d A tElectronic Record AssessmentDoes the application retain a record on durable electronic media (i.e., disk, tape, CD, network or other non-transient media)?Does the application create modify store archive or transmit a GxP record?Does the application create, modify, store, archive, or transmit a GxP record?

10


Electronic Signature AssessmentgAre signatures, initials, or other operator identification required for the operations documented by this application?Are decisions made on the information documented by this application prior to

t i i h d d t ?operators signing any hard-copy documents?

Section E: GAMP-5 Category Assignment1 I f t t S ft1 Infrastructure Software3* Non-Configured4 Configured

C5 Custom

*Category 2, from GAMP 4, was eliminated in the GAMP 5 revision

11


Assessment Conclusions

The application is determined to be GxP-related. Validation and controls appropriate for GxP-related applications apply.The application is determined NOT to be GxP-related. No additional controls are required by GxP.The application generates electronic records requiring the controls specified in 21 CFR Part 11.21 CFR Part 11.The application does NOT generate electronic records requiring the controls specified in 21 CFR Part 11.The application incorporates or requires an electronic signature for a GxP-related function. The controls specified in 21 CFR Part 11 apply.The application does NOT use or require an electronic signature for a GxP-related function.

12


Assessment Conclusions

GAMP-5 Category Assignment

1 Infrastructure Software

3 Non-Configured

4 Configured

5 Custom

13

Risk Identification and PrioritizationRisk Identification and Prioritization

Severity – Impact on patient safety, product quality and data integrityy p p y, p q y g yProbability – Likelihood of the fault occurringRisk Class – Determined by the relationship between Severity and ProbabilityProbabilityDetectability – Likelihood that the fault will be detected prior to harm occurringRisk Priority – Determined by the relationship between Risk Class andRisk Priority – Determined by the relationship between Risk Class and DetectabilitySuccessful execution of this method depends on the ability of the CSRA team to agree on the meaning of High, Medium, and Low for each ea o ag ee o e ea g o g , ed u , a d o o eacsegment of the assessment.

14


Guidance for Functional Risk AssessmentAssess each of the hazards associated with a function in two stages.Stage 1 – Severity of impact on patient safety, product quality and data integrity is plotted against the likelihood that a fault will occur, giving Risk Class.Stage 2 – Risk Class is then plotted against the likelihood that the fault will be detected before harm occurs giving a Risk Priority.

15


16


System or Data DestructionyDestruction of system due to power surgeLoss of data due to power outage/brown-outLoss of system access due to power outageLoss of system access due to power outageLoss of data due to storage faultLoss of system access due to processor or memory failureDestruction of system due to loss of environmental controlDestruction of system due to fireDestruction of system due to earthquake or other disastersDestruction of system due to earthquake or other disastersBackup/Restore procedure ineffective

17


SecurityyPhysical security breach of server/computerLogical security breach from outside the organizationLogical security breach from inside the organizationLogical security breach from inside the organizationComputer Virus infectionExecution of privileged functions by unauthorized personUntrained operators using the systemForgery of electronic signaturesCopying of electronic signaturesCopying of electronic signaturesTampering with completed recordsIncomplete electronic signatures accepted

18


Human FactorsReliance on (only) color for critical alarmsReliance on (only) audio for critical alarmsCritical faults do not require acknowledgementCritical faults do not require acknowledgementAlarm conditions not captured in permanent recordPerformanceSystem inability to service maximum number of concurrent usersOperation sequence impacted by system loadAlarms not provided to operators in real timeAlarms not provided to operators in real timeTime-critical events not serviced in time

19


LogicalgImproper user inputs or sequence corrupts or disrupts systemThroughput cannot meet demandOperators not informed of system or data failureOperators not informed of system or data failureResult algorithms incorrectSafetySystem fault creating an employee safety hazardSafety interlock fails to disable machineImproper wiring creates electrical hazardImproper wiring creates electrical hazardSystem SpecificList hazards specific to system functionality

20

SeveritySeverity

Characteristic Low Medium HighSeverity Cosmetic affect, fault forces

excess operator documentationAlarmed, readily recoverable

failure of a key system functionUnrecoverable or extended

failure of primary systemexcess operator documentation, occasional rejection of good product, momentary operator

intervention required to correct non-critical function

failure of a key system function, non-critical data loss, failure of a

minor specification.

failure of primary system function(s), severe regulatory

impact, critical data loss

Severity Expected to have a minor Expected to have a moderate Expected to have very significant negative impact. Damage would

not be expected to have long-term detrimental effects.

impact. Damage would be expected to have short to

medium term detrimental effects.

negative impact. The impact could be expected to have

significant long-term effects and potentially catastrophic short-

term effects.Severity Hazard is not expected to result Hazard could directly result in Hazard directly results in theSeverity Hazard is not expected to result

in negative medical consequences or any

complications.

Hazard could directly result in moderate injury to the patient or

operator

Hazard could indirectly affect the patient such that delayed or

incorrect information could result

Hazard directly results in the death or serious injury of the

patient or operator

Hazard indirectly affects the patient such that delayed or

incorrect information could result in moderate injury to the patient. in the death or serious injury to

the patientSeverity Hazard will cause small damage

to the businessHazard will cause considerable business or image damage, but will not endanger the company

Hazard will/is;

Endanger people

Contrary to law or regulation

21

Contrary to law or regulation

Damage to company image with unforeseeable consequences.

LikelihoodLikelihood

Characteristic Low Medium HighCharacteristic Low Medium HighProbability <1 incident per month <1 incident per week, but >1 per

month.Once or more per day

Probability Frequency of the event occurring is perceived to be once per ten

thousand transactions

Frequency of the event occurring is perceived to be once per

thousand transactions

Frequency of the event occurring is perceived to be once per

hundred transactionsthousand transactions thousand transactions hundred transactions

Probability Not expected to, or will rarely occur during the life of the

product/system under normal

Likely to occur infrequently or several times during the life of the

product/system under normal

Likely to occur regularly or many times during the life of the

product/system under normal operating conditions. operating conditions operating conditions

Probability ≥1:1001 – 5,000 =1:101 – 1,000 ≤1:100

Probability The problem will only occur if several events happen at the

same time

The problem couldn’t really be excluded for a long time, even

under normal conditions.

Failure will happen at regular intervals

22

DetectionDetection

Characteristic Low Medium HighDetectability Very difficult or nearly

impossible to capture the Some automated error

checking processes exist. O i

High level of error checking processes

i Oerror One-over-one review may be required. It’s likely that the error will be captured

in review of outputted information.

exists. One-over-one review required. Missed error will be obvious in

review of outputted information.

23


24

RA Form - ExampleRA Form Example

Project Title Example Project Number XX-XX-XXXXp

Scope

Risk Assessment

FunctionSub-

Function Comments

RelevanceGxP or

Business

Risk Scenarios

Probability of

OccurrenceSeverity Risk

Class Detectability Priority

A L L 3 L M

B L M 3 M LB L M 3 M L

C L H 2 L H

D M L 3 H L

EE M M 2 M M

F M H 1 L H

G H L 2 H L

25

H H M 1 M H

I H H 1 H M

Risk Based Test PlanningRisk Based Test Planning

Risk Level Testing Strategy

Zero Function is not related to a URS. No testing requiredNo testing required.

4 PQ testing only

3 Positive OQ testingIndirect PQ testingIndirect PQ testing

2 Positive OQ testingDirect PQ testing

High Positive and Negative OQ testingDi t PQ t tiDirect PQ testing

26

Risk Based Test Planning – ExamplesRisk Based Test Planning Examples

Function Low Risk Medium Risk High Risk

Input function with acceptable data range

of 10.0 – 20.0

Verify normal data is accepted

Boundary testing: 1 value below 10, 1 value in range, 1 value above

20.

Boundary testing: 9.9, 10.0, 10.1, 19.9, 20.0,

20.1

N ll l h ll N ll l h llNull value challenge Null value challenge

Incorrect decimal precision

Alpha character

Temperature control for an instrument

Verify calibration procedures

Verify accurate calibration throughout

operating range

Verify accurate calibration throughout

operating range

3 i t b d 6 i t b d3-point boundary testing for alarms

6-point boundary testing for alarms

Challenge control precision against defined process

27

pparameters

Periodic Reviews of RisksPeriodic Reviews of Risks

Change Control AssessmentsChange Control AssessmentsSystem UpgradesNew Interface(s)New Modules

28

Apply Risk Management to Computerized and Automated Systems

Health & Medicine

Transcript of Apply Risk Management to Computerized and Automated Systems