Aseptic Site Risk Assessment Tool” - PQRIpqri.org/wp-content/uploads/2015/11/Baumstein2.pdf•...
Transcript of Aseptic Site Risk Assessment Tool” - PQRIpqri.org/wp-content/uploads/2015/11/Baumstein2.pdf•...
Challenges and Solutions for the Quantification and Mitigation of Risk When
Assessing Global Aseptic Manufacturing Operations
“Aseptic Site Risk Assessment Tool” Presented by:
Michael Baumstein
Global Quality Operations
Microbiology and Aseptic Support (MAS) PQRI: September 17, 2014
Overview
• This slide deck is meant to provide a high-level overview
of the Aseptic Site Risk Assessment Tool (ASRAT)
process:
– Purpose & Scope
– ASRAT Process Flow
– Fundamental principals including benefits and limitations
– An explanation of the risk ratings and data analysis
• The MAS team currently is responsible for facilitating
both data collection and subsequent report generation:
– The MAS Team facilitates the process and provides
support to address any specific questions or comments
related to the program
What is the Purpose & Scope?
• Develop a tool to assess general aseptic manufacturing risk by site /
area.
• Designed to be simple, easy to use, applied consistently across the
network and account for site variability.
• Any tool should be able to assess risk based upon available internal
metric data.
• Even though questions posed may be retrospective (i.e. trailing
indicators) they should reflect prospective future risk.
• Can be used for indicated risk mitigation by sites with support by
MAS and other technical functions as necessary.
• This is a supplementary tool to enhance risk mitigation information
already available to sites through their current Quality Assurance /
Control process and procedures.
• ASRAT metrics are only one measure to consider the state of
site compliance to internal as well as external expectations.
Some Key Process Fundamentals …
• The risk algorithm employed was discussed and endorsed by the ASRAT Steering Team.
• Important to gain clarity and consensus on scope, risk scoring, and assignment of overall risk status.
• Process for data gathering needed to be efficient: 1. MAS request data from the sites (~30 days after close of the quarter)
2. Sites provided ample time to populate quarterly data (1-3 weeks)
3. MAS reviews and summarizes data (~1 week)a
4. MAS generates final report to Quality Leadership (< 1 week)b
• MAS shares draft quarterly data with the ASRAT Steering Teama.
• Final quarter-over-quarter progress report distributed to and results are shared with both site quality leaders and the Quality Operations Leadership Team (QOLT)b.
• Mitigation plan / action items are defined and discussed with ASRAT Steering Team.
What are the Key Process Activities?
• MAS posts updated spreadsheet file that is designed for shared access by a site data entry designee.
• Site designee enters data values into the file which automatically calculates program category metrics and applies risk threshold levels.
• Site specific data entries and calculations may be reviewed anytime during the open data entry period.
• MAS generates a summary report based upon data entered into the spreadsheet file.
• MAS reviews and finalizes the compiled summary report with input from the ASRAT Steering Team.
• Finalized report and any suggested mitigation(s) are approved by the ASRAT Steering Team.
How are Results Defined?
• Not all aseptic sites are equivalent…for example: – Different products, processes, equipment and facilities are found
throughout the network reflecting differing levels of potential risk.
• Ratings are not an absolute measure of site risk, rather they
are relative risk rankings.
• Risk thresholds have been set in a balanced manner in order
to better accommodate site differences.
• Metric data typically includes the total number of samples
collected and the number of deviation (i.e. Action Level)
events. – EM, Personnel Monitoring, WFI Testing and Pre-Filtration Bioburden
• ASRAT emphasizes trended / extended information not
specific deviations which are handled through established site
quality operation process / procedures.
So What are the Assessed Categories?
1. Process Performance – Media Fills
– Sterility Testing
– Pre-Filtration Bioburden
– Water (WFI) Testing
2. Environmental Performance – Microbiological Environmental Monitoring within Grade A & B areas
– Microbiological Environmental Monitoring within Grade C & D areas
– Surface
– Passive Air
– Active Air
– Microbiological Personnel Monitoring
There are two main topic indicators that are considered
and assessed using metric data within the ASRAT:
Additional ASRAT Process Details …
• Site-level risk ratings (i.e. Acceptable, Management Awareness,
or Management Alert) are assigned each quarter.
• Ratings are based in-part on the previous quarter’s overall ranking
as well as current results in each defined program category.
• Program categories are grouped by weighted risk as follows:
• Higher Risk Activities: Media Fill failures, Sterility Test failures
• Medium Risk Activities: Grade A/B EM Action Level excursions, Personnel
Monitoring Action Level excursions, WFI Test failures
• Lower Risk Activities: Pre-Filtration Bioburden Test failures, Grade C/D EM
Action Level excursions
• Higher Risk Activities have absolute risk thresholds (e.g. whole
number value assignments).
• Medium and Lower Risk Activities have normalized proportional
(%) thresholds based on historical performance in the network.
An Explanation on Risk Thresholds …
Risk Thresholds and how the ASRAT Deals with Site-
to-Site Variability:
• Formulated risk ratings are not an absolute measure of site risk.
– These are a combination of absolute risk from representative
program categories, normalized risk (percentage change for
medium and lower risk program categories) and the relative ranking
from previous quarter(s).
• Normalized risk thresholds are calculated based on 85th and 95th
data percentiles and updated (~2 years).
• Overall site risk rating is based on program category risk value and
weight as well as historical quarterly performance results.
• Data is reviewed quarterly by the dedicated ASRAT Steering Team
to provide additional details on the findings to support final site risk
rating assignments.
Assigned Risk Level Examples …
RISK LEVELS
PROGRAM Category Acceptable Management
Awareness
Management
Alert
Media Fill (Failures) 0 Failures 1 Failure >1 Failure
Media Fill (Positive Units) 0 Positives 1 Positive >1 Positive
Sterility Test (Failures) 0 Failures 1 Failure >1 Failure
Environmental Monitoring, Grades A/B Air
& Surface (% exceeding Action Level) <0.34% 0.34 to 0.7% >0.7%
Environmental Monitoring, Grades C/D Air
& Surface (% exceeding Action Level) <0.67% 0.67 to 1.7% >1.7%
Environmental Monitoring, Personnel
Monitoring on exit (% exceeding Action
Level)
<0.47% 0.47 to 1.1% >1.1%
Water (WFI) Testing (% exceeding Action
Level) <0.62% 0.62 to 2.0% >2.0%
Pre-Filtration Bioburden (% exceeding
Action Level) <0.6% 0.6 to 3.2% >3.2%
Further Detail on Risk Rating Rationale
• A Management Awareness (yellow) or Management Alert
(red) rating in a single medium or lower risk activity program
category will not cause the overall site level rating to move to
the Management Alert (red) risk level, regardless of the
previous quarter’s rating.
• However, a failure in a higher risk activity program category
(e.g. Media Fill with >1 positive unit or >1 Sterility Test
failure) or a combination of multiple medium and lower risk
activity program categories at Management Awareness
(yellow) or Management Alert (red) can result in an overall
Management Alert site level rating, even if the previous
quarter was Acceptable.
Example #1: Result Interpretation
• Overall rating from previous quarter was Acceptable (green).
• One Management Awareness (yellow) in a lower risk activity (Grade C/D EM).
• Overall rating for this quarter is Acceptable (green).
0
0
0
0.2%
1.1%
0.5%
0.0%
0.0%
Media Fill (Failures)
Media Fill (Positive Units)
Sterility Test (Failures)
Environmental Monitoring, Grades A/B Air & Surface (% exceeding Action Level)
Environmental Monitoring, Grades C/D Air & Surface (% exceeding Action Level)
Environmental Monitoring, Personnel Monitoring (% exceeding Action Level)
Water (WFI) Testing (% exceeding Action Level)
Pre-Filtration Bioburden (% exceeding Action Level)
0
0
0
0.1%
0.5%
0.3%
0.0%
0.0%
Example #2: Result Interpretation
• Overall rating from previous quarter was Acceptable (green).
• Two Management Awareness (yellow) in a higher risk activity (Media Fill).
• Overall rating for this quarter is Management Awareness (yellow).
Media Fill (Failures)
Media Fill (Positive Units)
Sterility Test (Failures)
Environmental Monitoring, Grades A/B Air & Surface (% exceeding Action Level)
Environmental Monitoring, Grades C/D Air & Surface (% exceeding Action Level)
Environmental Monitoring, Personnel Monitoring (% exceeding Action Level)
Water (WFI) Testing (% exceeding Action Level)
Pre-Filtration Bioburden (% exceeding Action Level)
0
0
0
0.2%
0.7%
0.1%
0.0%
0.0%
1
1
0
0.2%
0.3%
0.1%
0.0%
1.2%
Example #3: Result Interpretation
• Overall rating from previous quarter was Management Awareness (yellow).
• One Management Awareness (yellow) in a higher risk activity (Sterility Test), two Management Alerts
(red) in medium risk activities (Grade A/B EM and Personnel Monitoring) and two Management Alerts
(red) in higher risk activities (Media Fill).
• Overall rating for this quarter is Management Alert (red).
Media Fill (Failures)
Media Fill (Positive Units)
Sterility Test (Failures)
Environmental Monitoring, Grades A/B Air & Surface (% exceeding Action Level)
Environmental Monitoring, Grades C/D Air & Surface (% exceeding Action Level)
Environmental Monitoring, Personnel Monitoring (% exceeding Action Level)
Water (WFI) Testing (% exceeding Action Level)
Pre-Filtration Bioburden (% exceeding Action Level)
0
0
2
2.7%
0.0%
20.6%
5.0%
0.0%
2
220
1
5.6%
0.0%
10.5%
0.0%
0.0%
Report – Overall Site Rankings
Site
2QYear3 Data
Review
1QYear3 Data
Review
4QYear2 Data
Review
3QYear2 Data
Review
2QYear2 Data
Review
1QYear2 Data
Review
4QYear1 Data
Review
3QYear1 Data
Review
Site 1
Acceptable. Management Awareness. No
improvement in C/D EM results
and increased WFI test failure
rate.
Acceptable. Improved A/B EM
results. Watch C/D EM results.
Management Awareness. High
A/B and C/D EM results.
Acceptable. Watch C/D EM data
for return to poor trend.
Acceptable. No action. C/D EM
results have improved.
Acceptable. No action. Watch
C/D EM data for poor trend in next
Q.
Acceptable. No action. Watch
C/D EM data for poor trend in next
Q.
Site 2 Acceptable. Acceptable. Pre-filtration
bioburden continues increasing
trend.
Acceptable. Watch pre-filtration
bioburden for increasing trend.
Acceptable Acceptable Management Awareness. Media
fill failure.
Acceptable. Follow up on MSQA
observation.
Acceptable
Site 3 Acceptable. No data. Shutdown
for upgrade until 2013.
Acceptable. No data. Shutdown
for upgrade until 2013.
Acceptable. No data. Shutdown
for upgrade until 2013.
Site 4
Acceptable. Acceptable. Watch pre-filtration
bioburden for increasing trend.
Management Awareness. High
personnel monitoring results.
Acceptable Management Awareness.
Continued high personnel
monitoring results.
Management Awareness. Sterility
test failure and high personnel
monitoring results.
Acceptable. Dona to follow up on
PM and A/B EM data.
Uncertain. Need info on current
processes and specs. Sterile or
low bioburden? Regulatory
commitments? Dona to follow up.
Site 5
Management Awareness.
Increased personnel monitoring
results.
Acceptable. No sterility test
failures this quarter.
Management Alert. Finished
product sterility test failures (1).
In-process sterility test failures (5).
Management Alert. Finished
product sterility test failures (2).
In-process sterility test failures (3).
Management Awareness. WFI
test failures this quarter.
Management Alert. Continued
sterility test failures, but trend is
improving.
Management Awareness.
Concern with sterility test results.
Bill to contact Gustavo Scaffa, cc:
Chris Smith.
Management Awareness.
Concern with sterility assurance.
Bill to contact Gustavo Scaffa, cc:
Chris Smith.
Site 6 Acceptable. Acceptable. Acceptable. Acceptable. High C/D EM results. Acceptable. Management Awareness. Two
media fill failures this quarter.
Acceptable Management Awareness. Paolo
working with site to resolve media
fill positives.
Site 7
Management Alert. High A/B EM
and personnel monitoring results.
Management Alert. Finished
product sterility test failures (3).
High A/B EM and personnel
monitoring results.
Acceptable. However EM results
not available by grade. Will be
corrected with implementation of
MODA system. Watch high in-
process sterility test failures.
Management Awareness. Finished
product sterility test failure (1).In-
process sterility test failures (287).
EM results not available by grade.
Will be corrected with
implementation of MODA system.
Acceptable. However EM results
not available by grade. Will be
corrected with implementation of
MODA system.
Acceptable. However EM results
not available by grade. Will be
corrected with implementation of
MODA system.
Uncertain. Paul will follow up on
missing EM and PM data.
Management Awareness. Paul
working with site to resolve issues.
Site 8 Acceptable. Acceptable. Acceptable. Acceptable. Acceptable. Acceptable Acceptable Acceptable
Site 9 Management Awareness.
Increased personnel monitoring
results.
Management Awareness.
Increased personnel monitoring
results.
Acceptable. Acceptable. Acceptable. Management Awareness.
Continued media fill failures.
Management Awareness. Dona to
review media fill failure.
Acceptable
Site 10 Acceptable. Acceptable. Acceptable. Acceptable. Acceptable. Acceptable Acceptable Acceptable
Site 11 Acceptable. Acceptable. Acceptable. Acceptable. Acceptable. Uncertain. No data. Uncertain. All zero results raise
doubts. Stew to review MSQA
audit.
Uncertain.
Site 12 Acceptable. Acceptable. Acceptable. Acceptable. Acceptable. Uncertain. Uncertain. Uncertain.
Site 13
Acceptable. Acceptable. Acceptable. Acceptable. Acceptable. Uncertain. All zero results raise
doubts. Bill to contact Uma.
Review EM program?
Uncertain. All zero results raise
doubts. Bill to contact Uma.
Review EM program?
Uncertain. All zero results raise
doubts. Bill to contact Uma.
Review EM program?
Site 14
Acceptable. Acceptable. Acceptable. Management Awareness. Positive
unit (1) in finished product media
fill and high C/D EM results.
Acceptable. Acceptable Acceptable. Follow up on existing
programs satisfactory.
Uncertain. All zero results raise
doubts. Stew to review EM
program.
Site 15
Management Awareness. Positive
unit (1) in finished product media
fill.
Acceptable. Acceptable. Acceptable. Acceptable. Acceptable. Watch media fill
results next quarter.
Acceptable Acceptable
Use of Data Trending Visual Cues ...
Trend
Indicator:
J = Improving
K = No Change
L = Worse
PROGRAM Category Site A Site B
Media Fill (Failures) 0 K 0 K
Media Fill (Positive Units) 0 K 0 K
Sterility Test (Failures) 0 K 3 L
Environmental Monitoring, Grades A/B Air & Surface
(% exceeding Action Level) 0.22% J 0.44% L
Environmental Monitoring, Grades C/D Air & Surface
(% exceeding Action Level) 0.82% J 1.47% L
Environmental Monitoring, Personnel Monitoring on
exit (% exceeding Action Level) 0.82% L 0.09% J
Water (WFI) Testing (% exceeding Action Level) 2.05% L 0.05% K
Pre-Filtration Bioburden (% exceeding Action Level) 0.77% J 0.27% J
Some Final Thoughts on ASRAT …
• The program’s initial development to actual
implementation took about a year.
• Has served as a novel way to assess prospective future
risk beyond simple reactionary limits (i.e. Alert/Action
levels).
• Provides additional management awareness to site
performance and overall site “health”.
• Adaptability via risk threshold baseline adjustments.
• Active site engagement makes this process at times
challenging, certainly value added and most importantly
provides for continuous site improvement