Post on 17-Jan-2016
1
Validation of Qualitative Microbiological Test Methods
NCS Conference
Brugge, October 2014
Pieta IJzerman-Boon (MSD)Edwin van den Heuvel (TUe, UMCG/RUG)
2
Contents
• Introduction
• Statistical Detection Mechanisms
• Validation Issues
• Likelihood-Based Inference
• Conclusions
3
Introduction
Validation parameters Qualitative tests
Microbiological guidelines
Analytical guideline
Accuracy and precision EP
Repeatability USP
Specificity EP/USP ICH
Detection Limit EP/USP ICH
Ruggedness USP
Robustness EP/USP
• Guidelines on validation do not agree
4
Introduction
• In this presentation we will show an optimal validation strategy:–Compare methods–Two dilutions–Optimal densities for the two dilutions–Required number of samples
• Optimal validation strategy differs substantially from the guidelines
5
Statistical Detection Mechanisms
• Suppose a test sample is tested with a qualitative test
• The sample contains X organisms–X=0: sample is sterile–X>0: sample is contaminated
• The outcome of the test is Z –Z=1: positive test result–Z=0: negative test result
6
Statistical Detection Mechanisms
• Classification of test result
• So we need to look at the conditional probabilities
• The function describing this detection probability is referred to as the detection mechanism
Okay
False Positive
False Negative
Okay
Number of Organisms
X=0 X>0
Tes
t R
esul
t
Z=
0Z
=1
xX|ZPx 1
7
Statistical Detection Mechanisms
• Zero-deflated binomial mechanism:
– is the false positive rate: (0)=–p is the detection proportion: if =0 then it is the
probability to detect just one organism: (1)=p–If =0 and p=1 the test method is perfect– and p are related to specificity and accuracy–The binomial mechanism (=0) was introduced in
Van den Heuvel and IJzerman-Boon (2013)
0if111
0if
xp
xx x
8
Statistical Detection Mechanisms
=0, p=0.85 =0.10, p=0.70
9
Validation Issues
• Estimate detection mechanism via experiments–Exact low spikes of X cannot be generated–Hence the detection probability (x) cannot be
estimated, only the average proportion over samples
• Expected proportion of positive test results:–Assume that the number of organisms X ~ Poi(
peX 11E
10
Validation Issues
• The detection proportion p cannot be estimated–Without knowledge on the average number of
organisms in the test samples–With serial dilution experiments
• The false positive rate can always be estimated using samples from a blank dilution (=0)
Compare alternate with compendial method–Using the same for both methods–Likelihood ratio test (LRT)
11
Likelihood-Based InferenceExperimental Design
• Suppose we test samples from the same dilution with two methods–Alternate method: i=1–Compendial method: i=2–Dilution has on average organisms per sample
–Number of samples tested per method: n
• Expected proportion of positive results now depends on method i (i=1,2):
ipii e 11
12
Likelihood-Based InferenceExperimental Design
• Asymptotic distribution of LRT for comparing these proportions converges to -distribution with
• Hence, power can be optimized by maximizing–Bacterial density can be optimized independently
from sample size n–There is a single optimal density
)nc(21
))((
)(nnc
2121
221
2
2
nc
13
Likelihood-Based InferenceExperimental Design
Compendial: 2=0.01p2=0.95
14
Likelihood-Based InferenceSimulations
Simulation Results: Single dilution • Average density • Detection proportions pAL=0.7 and pCM=1
• Power (%) of likelihood ratio test LRT for differences in detection probabilities for various false positive rates
AL=CM=0 AL=0.05, CM=0 AL=CM=0.05
n=150 n=200 n=250 n=150 n=200 n=250 n=150 n=200 n=2501.01.52.02.53.0
65.369.270.167.564.8
74.680.481.880.476.8
85.888.589.689.284.4
49.157.360.460.257.8
56.168.572.371.971.9
68.579.682.782.879.8
61.067.267.363.961.3
71.777.379.077.375.0
83.286.187.885.982.9
15
Conclusions
Optimal strategy when parameters are unknown–Compare alternate with compendial method–Two dilutions are needed
1. Blank dilution
2. Dilution with on average ~2 organisms–Sample size should be at least n=200
• False positive rates can be tested with LRT
• Accuracy pAL/pCM can be tested with appropriate CIs as an alternative for the LRT for the ratio pAL/pCM
(IJzerman-Boon and Van den Heuvel, 2014)
16
Conclusions
• Differences with guidelines–Only specificity and accuracy need to be considered–Two dilutions are needed, using five 10-fold dilutions
is a loss of power–The optimal density is ~2 CFU/unit, ~5 CFU/unit is
much too high–Use 200 instead of 5 samples per method and dilution
to detect a 30% drop in accuracy with 80% power
17
References
• IJzerman-Boon PC, Van den Heuvel ER, Validation of Qualitative Microbiological Test Methods, Submitted, 2014.
• Van den Heuvel ER, IJzerman-Boon PC, A Comparison of Test Statistics for the Recovery of Rapid Growth-Based Enumeration Tests, Pharmaceutical Statistics, 2013; 12(5): 291-299.
• EP 5.1.6 Alternative Methods for Control of Microbiological Quality
• USP <1223> Validation of Alternative Microbiological Methods
• ICH Q2 (R1) Validation of Analytical Procedures