Risk-Based Commissioning & Qualification Benchmarking
Educational Track #1
AgendaPart 1 (8:30 – 10:00)
• ISPE Baseline® Guide Volume 5: Commissioning & Qualification vs. ASTM E2500 – Mr. Robert Chew
• Overview of C&Q approaches – Abbott, Lilly, Pfizer
Break – Vendor areas open (10:00 – 10:30)
Part 2 (10:30 – Noon) • Panel Q&A
Wrap-up
ISPE Baseline Guide Volume 5: Commissioning and Qualification, vs. ASTM E2500
Robert E. Chew, PE
President, Commissioning Agents, Inc.
My Background in Subject
• Official trainer for ISPE Baseline Commissioning and Qualification Guide
• Member of team that wrote ASTM E2500
Key Points to Compare
• Terminology
• Qualification practices
• Focusing the effort
• Roles and responsibilities
• Managing changes during the project
Terminology
• C&Q Baseline guide• Commissioning• Qualification
• ASTM E2500• Verification
Qualification Practices
• C&Q Baseline Guide• “To meet the demands of the regulators”
• ASTM E2500• Not mentioned
Focusing the Effort
• C&Q Baseline Guide• Impact assessments
• Direct impact systems and critical components
• V-model
• ASTM E2500• Risk assessment -> critical aspects
Roles and Responsibilities
• C&Q Baseline Guide• Quality pre and post approves protocols; quality control
role
• ASTM E2500• Quality involved in process requirements, risk
assessments, critical aspects, and acceptance and release
• Subject matter experts used throughout
Managing Change
• C&Q Baseline Guide• QA pre-approved change control
implemented at start of IQ/OQ
• ASTM E2500• QA pre-approved change control
implemented at start of manufacture of product for distribution
Common Elements
• Good engineering practices
• Planning
• Requirements definition
• Engineering design reviews
• Project change management
• Documented inspections and testing
Risk-Based Commissioning & Qualification Approach
Abbott LaboratoriesGlobal Pharmaceutical Operations
Steve Connelly
Overview
Where Abbott Is Today
C&Q Approach
Commissioning & Qualification Activities
How We Manage Cost
Lessons Learned
Where Abbott is TodayWhere Abbott is TodayRisk Based Maturity Model*Risk Based Maturity Model*
*Adapted from 2007 ISPE Tampa Conference Presentation
Commissioning & Qualification Approach C&Q Approach:
• Application of ISPE Baseline® Guide Volume 5: Commissioning & Qualification (last 2 years)
• Little to no implementation of ASTM E2500-07*• Level 3 (Risk-based Maturity Model)
Implementation:• Corporate policies support Science & Quality Risk
Management and includes Good Engineering Practice (GEP) reference
• Training ( RBA, GEP, ECM. Commissioning, Risk Assessments)
• Tools and Templates • Discretion of the Validation Review Board (VRB)
*ASTM E2500-07: A Standard Guide for the Specification, Design, & Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment
Science & Quality Risk
Management Qualification
Process Flow
START
Does system directly impact
product quality?
Does system support a direct impact system?
Design Qualification
Installation Qualification
Evaluate Risk/Impact
Assessment
VCR Submittedw/Assessment
Report and Rationale
System have operational functions?
Yes
Operational Qualification
Support of CPPs tested in
PV?
Performance Qualification
CommissioningPlan
Pre-Qualification Engineering tests /FAT as applicable
VCR Submittedw/Assessment
Report and Rationale
Process Complete
VCR Closed
VCR Submittedw/Assessment
Report and Rationale
User Requirements
Functional Design
No
Yes
No
Yes
No
Yes
Engineering tests /FAT as applicable
CommissioningPlan
No
Direct Impact
Systems
Indirect Impact
Systems
No Impact
Systems
Good Engineering Practices Formal acknowledgement of GEP in corporate policy
Divisional GEP guideline consists of minimum requirements
Additional divisional guidelines are in development for life-cycle management
– Calibration
– Maintenance
– Engineering change management
Training to aid site support (implementation)
Requirements Based On Impact Category
*System = equipment, facility, utility
Three methods:–ISPE-Style System* Impact Assessment Questionnaire –Risk Assessment with any standard method (FMEA, HAZOP, etc.)
–Technical Evaluation by SMEs
•Documented and VRB approved•Determining impact to identifying critical elements
Quality Risk Control–Risk Assessment (i.e. FMEA, etc) used to enable:•Qualification focus on critical functions and/or components•Leverage limited commissioning testing (FAT/SAT) and •Eliminate some redundant activities (i.e. alarms, etc)
Commissioning When supporting Qualification, Commissioning must
include or address:• Engineering Change Management (ECM)
– After point commissioning data intended to support Qualification
• Testing– Methods described in sufficient detail for repeatability– Documentation confirms critical elements met criteria– Acceptance criteria documented prior to execution– Criteria based on User Requirements– Documentation of observed results
Commissioning & Qualification (C&Q) Plans typically contain run rules for project
Qualification – Equipment User requirements
• Product and Process User Requirement (PURS)• General User Requirements (GURS)
Commissioning (in support of qualification)• Test it once (or one set of multiple test repetitions, where
appropriate); document it correctly• Primarily installation activities, operational testing
leveraged in some instances (VRB discretion);compile, summarize test results for VRB approval
Qualification • When approved, commissioning reports complete the
IQ/OQ phase of Qualification with no additional “protocol” testing
Qualification – Software (APC/HMI/DCS)
Categorization (based on product quality impact)• Managing control issues for critical elements
• Supplier Assessments for direct impact systems only
• VRB discretion with technical advisory from Subject Matter Expert
Holistic Approach • Integrated Qualification document (IOQ for
equipment/controls; as applicable)
• APC specific qualification requirements
– Traceability matrix = Design Qualifications (DQ)
– Commissioning = Qualification except critical items
Qualification – Facilities
Indirect Impact Facilities - Commission
– Design and construction conforming to GEP
– Abbott Engineering Standards
Direct Impact Facilities – Commission/Qualify
– Meet quality specifications (i.e. Aseptic facility, etc)
– Design and construction conforming to GEP
– Abbott Engineering Standards
Qualification – HVAC Categorization
• Interdependent with facility quality attributes
– For example: Continuous monitoring for critical environmental aspects; therefore facility and HVAC can be Indirect Impact (HVAC controls would then be qualified)
– For example: Temperature, RH, and room pressure differentials typically critical elements which are monitored and recorded continuously, alarming when parameters exceed specifications; VRB discretion as to qualify
Qualification – Utilities
Impact Categories - Typically critical or non-critical
Utility systems typically not big source of Validation Change Request (VCR) sources
Only regulatory requirement concern is typically Potable Water feed to purification
– Purpose of pretreatment usually to preserve life of purification equipment rather than direct impact of water quality
How We Manage Cost…$ In 2007, reviewed C&Q costs for several capital projects
to:– Establish a baseline – Compare costs for the “traditional, validate everything” C&Q
approach versus the risk-based C&Q approach
$ Cost comparisons across projects difficult because– Inconsistent reporting and project scope varies widely
$ Drive cost reporting model for capital projects needed to: – Address scope differences, track additional parameters:
Number of Qualification Protocols per project Number of Qualified Systems Total Number of Systems
– Three parameters documented in C&Q Plan
• Monthly Capital Project Status Reports are used to track costs and progress
• There is a wide variation in the tracking systems used by the engineers for the project costs
• Most project engineers / project managers use spreadsheets of their own design to track costs
How We Manage Cost…Commissioning & Qualification Cost Tracking Details
Opportunities Identified for Consistent Reporting
Engineering/Commissioning/Qualification costs categorized below
Activities not linear, categorization to clarify “gray” areas of cost reporting
Defines appropriate cost “bucket” for consistency across projects
How We Manage Cost…
ENGINEERING COMMISSIONING QUALIFICATION (CERTIFICATION)FEP 1-3 Risk Assessment Validation PlanUser Requirements Failure Mode Effect Analysis Design QualificationFunctional Specification (Draft) I/O checkout Equipment Installation QualificationPreliminary Design Calibration Equipment Operational QualificationDetailed Design Functional Specification (Revisions) Computer Hardware Installation QualificationConstruction Factory Acceptance Testing Computer Software Installation QualificationLoop Tuning Startup Controller Modification QualificationEquipment Installation/Alignment Trouble Shooting HVAC Installation/Operational Qualification
Sequence Testing Utility Installation QualificationAs-Builts (Non Qualified Systems) Process Qualification
Validated DrawingsSamples/TestingCleaning/PassivationInspections
Engin
eering W
ork
Ord
ers
/ C
ontr
acts
Lessons Learned
Risk-Based C&Q is an iterative process (one tiny step at a time)
Corporate & divisional policies in place, tools developed, although success for various sites, concern about corporate & regulatory audits
Sites want to maintain autonomy with guidance; common elements, not same approach (i.e. all engineers comply however variations due to specialty)
Application: Site readiness tools - identifying opportunities for closing gaps
Lessons Learned
Listen, Speak, Deliver. Communication is key.
Quality may wonder: What if engineering doesn’t do its job?;ensure compliance
Engineering may wonder:Quality = police & spell checkers; “right” the first tyme
Application:
Regular/frequent communication with VRB & project team
Lessons Learned
Sustainable Paradigm Shift
Sites: Implementation, accountability & ownership a must for sustainability
Resources: People, time & money, site retaining technical knowledge
Application: Training & support implementation as applicable
Thank you!
• Questions will be addressed during the Panel Q&A session (after break)
Eli Lilly & Company
Commissioning and Qualification Approach
Eli Lilly & Company Approach to Commissioning & Qualification
• C&Q Delivery Strategy
• C&Q Metrics
• C&Q Lessons Learned
• Improved C&Q Delivery Strategy
Recent GFD C&Q Delivery Strategy~ Baseline Guide approach (’02 – ‘07)
• Developed procedures governing C&Q activities
• Fully commissioned all assets ~ GEP
• Reserved qualification for critical aspects of DI systems only
• Leveraged commissioning documentation to support qualification – commissioning subject to GDPs
• Involved Quality at C&Q strategy level
• In addition, • Simulation has been used to commission automation aspects• CSV testing has been integrated within C&Q test documents
Fundamental C&Q Delivery Strategy~ for Global Facilities Delivery (GFD) Capital Projects
Profile
RequirementsProcurement
Design
Qualification
Commissioning
Construction/Installation
Validation
Planning
Operations/ Maintenance
Project
Owner
System Classification
Component Classification
DQ/DR
Simulation/ Offline Testing
RV
IV
FAT
SATFT
IQ OQ PQ
Initial Cals
Startup
Commissioning Package
Construction QA
Approve CQMP
Approve System-level Plans
Loops
This strategy has resulted in:
• Earlier resolution of issues
• Opportunity to understand systems
• Reduced Qualification time
• Reduced Qualification costs
• Fewer Qualification comments/discrepancies
C&Q Delivery Strategy ~ Results
C&Q Metrics
• Schedule Metrics
• Cost Metrics
• Discrepancy Metrics
Typical C&Q Schedule Metrics ~ Durations
• 10 months from facility mechanical completion to OQ completion
• Total Commissioning duration: 24 mos. (172 days/system)
• Total IQ/OQ duration: 6 months
• FT duration/system: 42 days
• IQ duration/system: 11 days
• OQ duration/system: 17 days
• Average # of days from MC to OQ completion: 227 days
Note: Metrics for new powder filling facility consisting of approx. 40 systems
C&Q Cost Metrics ~ Historical view
No C&Q prog1998- 2001
Full C&Q prog2002- pres.
Part. C&Q prog2001-2003
C&Q Costas % TIC
40%
20%
10%5%
25%
10%5% GOAL3%
C&Q Budget Metrics ~ capital projects completed in ‘07
Total Installed Cost C&Q Cost (% TIC)$319MM 6.0%
$186MM 1.5% (comm. only)
$180MM 5.5%
$145MM 5.7% (incl. non-CQ activities)
$14.5MM 4.3%
$11.6MM 4.3%
$7.2MM 2.2%
Average: 4.7% (C&Q projects only)C&Q delivery strategy has resulted in a step change in C&Q savings!!
Lilly C&Q Work Breakdown Structure
Lilly budgets/tracks C&Q expenses via 4 primary WBS categories:
• Planning & Design• Commissioning• Qualification• Administration
New C&Q Cost Data spec recently approved to provide guidance
Planning & Design - Subcategory
• C&Q plans and strategy• Initial schedule development• Estimating• System classification• Component classification• System C&Q plans• Vendor assessments• User requirements• Design review, spec review, drawing review reviews/tag list• Engineering requirements• Design Qualification• Traceability Matrices
Commissioning - Subcategory
• FAT development• FAT execution• Receipt Verification development• Receipt Verification execution• Installation Verification development• Installation Verification execution• SAT development• SAT execution• Construction turnover packages
• Start-up protocol development• Start-up protocol execution• Functional testing development• Functional testing execution• Commissioning packages• C&Q plan summary reports• FAT travel• Supporting documents
Qualification - Subcategory
• IQ development• IQ execution• OQ development• OQ execution• PQ development• PQ execution
Administration - Subcategory
• Training• Project management• C&Q resource travel• Administrative assistance• Audits• CSV support
Typical C&Q Budget Metrics ~ Cost Breakdown (%)
URs 3%
Procedures 2% Doc Revs 1%
Planning 5%IV 7%
SU 9% RV 10%
CSV Support 3%
Construction QA 4%
DQ/EDR 3%
CCs 2%
Impact Assessment 1%RI
1%
Training1%OQ 2%
IQ 2%Doc mgmt 12%
Project mgmt 11%
FT 11%
Travel 10%
Scheduling 0%
SAT1%FAT
1%
Note: Only includes cost data for C&Q contractor
C&Q Discrepancy Metrics ~ Philosophy
C&Q delivery strategy based on risk mitigation approach
• DR/DQ reduced risk during Construction/Build, Commissioning, Qualification
• Construction quality assurance activities reduced Commissioning risks
• Simulation reduced risk during Commissioning (Startup, SAT, FT)
• Commissioning (RV, IV, FAT, SAT, FT) reduced risk during subsequent Commissioning activities and Qualification
• As risk decreased, the number of discrepancies also decreased - resulted in cleaner qualification documentation and better schedule adherence during qualification
Typical C&Q Discrepancy Metrics
1230 discrepancies resulting from Construction QA activities 2800 discrepancies during commissioning 130 discrepancies during qualification
0
100
200
300
400
500
600
700
RV SIM FAT IV SU SAT IL IC FT CP IQ OQ PQ CQP US
C&Q Activities
# o
f D
isc
rep
an
cie
s
Typical C&Q Discrepancy Metrics ~ Categories
0
100
200
300
400
500
600
700
800
900
# o
f D
iscr
ep
an
cie
s
Accepted
Corrective Action
1. Properly define User Requirements Testable, verifiable, applicable
Not too prescriptive, no design details, no guesses
Include C&Q engineers in review/approval process
2. Clearly define data and documentation requirements
3. Change management Don’t implement formal change control too soon
Don’t confuse change control with version control
Ensure commissioning change process is well understood
Ensure construction changes are communicated to team
Ensure automation changes are reflected in design docs
C&Q Lessons Learned
Current GFD C&Q Delivery Strategy~ Enhanced Baseline Guide approach (last 2 years)
• Conducted several six sigma initiatives to improve C&Q processes
• Developed additional procedures and best practices pertaining to C&Q• GDP SOP• Change Management SOP• Transfer of Care, Custody and Control SOP • Discrepancy Management Best Practice
• Developed new specifications governing the following
• Vendor data and documentation• FAT/SAT• C&Q cost data
• Defined new project information management role
• Implementing new requirements definition process based on systems engineering principles
• Implementing formal Construction QA program
Current GFD C&Q Delivery Strategy~ Enhanced Baseline Guide approach (cont.)
• Developed Automation Lifecycle Management engineering standard to accommodate integration of CSV and C&Q resulting in:
• Elimination of duplicate requirements, design reviews, testing• Elimination of “all or nothing” approach to software testing• Allowance for risk-based delivery strategies Leverage offline or simulation testing to
meet requirements
• Assessing risk in areas beyond product quality to streamline C&Q delivery
• Vendor risk• Technology risk• Automation strategy
• Leveraging vendor testing to support commissioning vs. just using as contractual gate/risk mitigation
The resulting C&Q delivery strategy will hopefully be realized as another step change in C&Q savings! Potential C&Q costs as low as 3% of TIC
Questions?
Q&A
Risk Based Commissioning & Qualification Benchmarking
Presented by:
Nick Andreopoulos Senior Manager/Team leader
Pfizer Global EngineeringPeapack, NJ
Chris BeltzSenior Manager/Team Leader
Pfizer Quality OperationsKalamazoo, MI
Agenda
• Basis of C&Q Practices
• C&Q Program Management Approach
• C&Q Cost Management
• Key Lessons
Current C&Q Practices
• Primarily based on C&Q Baseline® Guide 5 • Certain sites have streamlined C&Q process
• C&Q “Lite” version for small projects and minor equipment changes
• Leveraging of Commissioning tests into IOQ• Applying some ASTM concepts – Focusing IOQ on risk to
product quality/patient safety• Functional Level Impact Assessment - Replacing
Component Level Impact Assessment• Currently developing and piloting full ASTM E2500
based Verification approach at targeted sites
C&Q Program Management
• C&Q supported jointly by Quality, Engineering and Automation/IT
• Quality requirements defined in Corporate Quality Standards
• C&Q Approach defined in Global Practices• Sites develop their own procedures and
validation master plans based on the global documents
C&Q Cost Management
• C&Q costs that are associated with equipment qualification are included in Project Capital budgets. Process Validation and Cleaning Validation are not included in project budgets
• C&Q cost managed by Project team
Key C&Q Lessons Learned• Upfront involvement by key stakeholders in the
Assessment steps to define the appropriate scope for qualification
• IOQ scope should be focused on critical aspects/functions. All other testing is GEP
• Testing documents should be based on SME execution and leveraged, where possible, from vendors. Level of documentation should be commensurate with risk to product quality/patient safety.
• Roles/Responsibilities need to be clearly defined at the outset of projects
Risk-Based Commissioning & Qualification Benchmarking
Educational Track #1
Part 2
AgendaPart 1 (8:30 – 10:00)
• ISPE Baseline® Guide Volume 5: Commissioning & Qualification vs. ASTM E2500 – Mr. Robert Chew
• Overview of C&Q approaches – Abbott, Lilly, Pfizer
Break – Vendor areas open (10:00 – 10:30)
Part 2 (10:30 – Noon) • Panel Q&A
Wrap-up
Thank You!!
• Robert Chew, Commissioning Agents• Matt Warhover, Commissioning Agents• Steve Connelly, Abbott Laboratories• Dave Dolgin, Abbott Laboratories• Scott Hamm, Eli Lilly• Rick Gunyon, Eli Lilly• Nick Andreopoulos, Pfizer • Chris Beltz, Pfizer• Great Lakes Chapter Board and Directors• Ron Dunn, GLC Manager
Top Related