Post on 04-Jan-2016
description
IPH
Accreditation & ValidationJoris Van Loco
Scientific Institute of Public HealthFood Section
Method Validation
Is method validation• analyzing 6 samples ?• Calculating the bias, repeatability,
reproducibility,… of a method ?• Knowing the detection limits of the method ?• knowing the uncertainty associated with a
method?• satisfying ISO 17025 assessors?
What is Method Validation?
Method validation is the process of proving that an analytical method is acceptable for its intended purpose
Why is Method Validation Necessary? To prove what we claim is true To increase the value of test results To justify customer’s trust To trace criminals
Examples• To value goods for trade purposes• To support health care• To check the quality of drinking water
When and How should Methods be Validated New method
development Revision of established
methods When established
methods are implemented in new laboratories
Interlaboratory Comparison
Single lab validation• Full Validation• Implementation Validation
Method performance parameters are determined using• equipment that is:
• Within specification• Working correctly• Adequately calibrated
• Competent operators
Validation and Quality Control
In house validation• (Bias), recovery• Repeatability• Within lab
reproducibility
Internal QC• Control charts
Starting data
Proficiency Testing• Bias (trueness)
Collaborative trial• Reproducibility• Bias (trueness)
Long term within lab reproducibility
Method Validation
Accuracy• Trueness (CRM)• Recovery (spikes)
Precision• Repeatability• (Within) reproducibility
Selectivity (& Specificity) Detection capability
• LOD, LOQ, CC, CC Linearity – calibration range Robustness
• Applicability – stability
Method ValidationPerformance Characteristics 2002/657/CE
Detection Capability
CCß
Decision Limit CC
Trueness/ Recovery
Precision Selectivity/ Specificity
Applicability/ Ruggedness/
Stability
S + - - - + + Qualitative methods C + + - - + +
S + - - + + + Quantitative methods C + + + + + +
S: Screening methods
C: Confirmatory methods
Linearity
Purpose• to evaluate the linear response of your instrument
How• Evaluating your calibration model
• Mandels fitting test• Lack-of-Fit• Residuals
Conclusion• Linear model • <> other (i.e. quadratic) regression model
Simple regression: confidence and prediction interval
-0,02
0
0,02
0,04
0,06
0,08
0,1
0,12
0,14
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5
concentration (ng/ml)
AU
Linearity
Residual plots (ei)
• with
Statistical tests• Lack-of-fit• Mandel’s fitting test
Linear relationship
-12
-8
-4
0
4
8
12
0 20 40 60 80 100
X
Re
sid
ua
ls
iii YYe ˆ
ii bXaY ˆ
Curvilinear relationship
-20
-10
0
10
20
0 20 40 60 80 100
X
Re
sid
ua
ls
Curvilinear relationship
-20
-10
0
10
20
0 20 40 60 80 100
X
Re
sid
ua
ls
Coefficient of correlation (r)
Is NOT a suitable measure for linearity
0
10
20
30
40
50
60
70
80
0,9975 0,998 0,9985 0,999 0,9995 1
correlation coefficient (r)
F m
and
el's
fit
tin
g t
est
Pb
Cd
Matrix Effect
Purpose• To evaluate whether you have
a concentration dependent systematic error due the matrix
• i.e. ion suppression How
• comparison of standard curve with matrix matched standard curve
Conclusion• Standard solutions, spiked
extracts or spiked samples for the calibration line.
4
)2()2(
11)(
22
2
,
2
,
as
aasa
aaissi
as
nn
SnSnSp
XXXXSp
bbbt
-0,05
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
0 50 100 150 200 250
addition
standard
Detection limit
DIN 32645 from blanks
from calibration data
Funk dynamic model
IUPAC
Coleman recursive formula
explicit formula
2
D x 0 f;αx
1 1 xx s t
m n Q
LD f;α
s 1 1x t
b m n
2
c y n2
ii 1
0 x1y b s t 1
nx x
2
y cD n
22i
i 1
s t y y1x 2 1
a na x x
c 1 α,v 0S t sαβ,v 0 1 α,v 0
D
δ σ 2t σK Kx
A I A I
B A1 α,v
0
σ σK 1 r(B,A) t
σ A
2
A1 α,v
σI 1 t
A
11
2 22 2D
D n 2,1 α n 2,1 βxx xx
x xs 1 x 1x t 1 t 1
a n S n S
1
2 2
D H V
J J 4HKx DL
2H
n 2,1 β
aA
s t
12 2
n 2,1 α
n 2,1 β xx
t 1 xB 1
t n S
2xx
1F B 1 S
n
xxG 2AB S 2 1xxH A S J G 2x 2K F x
Detection Limits
A) DIN 32645
Detection limit
by fast estimation:
Capability limit
Determination limit
by fast estimation
Factor for fast estimation
2k
D x0 f;ax
y - a 1 1 xx = = s t + +
b m n Q
n;α x0Dx = 1,2 Φ s
2
C NG x0 f;βx
1 1 xx = x + s t + +
m n Q
2DDT x0 f;α
x
x - x1 1x = k s t + +
m n Q
n;α x0DTx = 1,2 k Φ s
n;α f;α
1Φ = t 1+
n
Detection Limits
B) Funk
Detection limit dynamic model
Determination limit dynamic model
2
c y n2
ii 1
0 x1y b s t 1
nx x
2
y cD n
22i
i 1
s t y y1x 2 1
a na x x
2y
c n2
ii 1
s t 1 xx 1
a nx x
2
ch y n
2
ii 1
x x1y b 2 s t 1
nx x
2
y hhDT n
22i
i 1
s t y yy b 1x 1
a a na x x
Detection Limits
C) IUPAC
Detection limitc 1 α,v 0S t s
αβ,v 0 1 α,v 0D
δ σ 2t σK Kx
A I A I
B A1 α,v
0
σ σK 1 r(B,A) t
σ A
2
A1 α,v
σI 1 t
A
Detection Limits
Detection limits “How to”
Choose a definition and stick to it • Describe the equation used in the validation file• Problems
• statistics <> practical limitations• statistics <> ID-criteria
Practical LOD• Analyzing samples with decreasing concentration• Minimum concentration which fulfills the identification
criteria = practical limit of detection• Repeat the experiment
S/N• i.e. LOD=3xS/N
Quantitation Limit (LQ)
The quantification limit is the minimum signal (concentration or amount) the can be quantified. • the residual standard deviation (RSD) is
included in the definition.
• The IUPAC default value for RSDQ= 0.1 (or 10%). LQ=10sQ.
Q RSDL 1
- and -error
-error• risk of erroneously rejecting H0
• i.e. risk of the conviction of an innocent -error
• risk of erroneously accepting H0
• i.e. risk of the non conviction of a criminal
Detection CapabilityCase of a permitted limit (MRL)
MRL CC
Signal orConcentration
CC
=
+1.64MRL +1.64sample
Determination of CC and CC with ISO 11843Plot of Fitted Model
Conc (µg/kg)
un
corr
ect
ed
re
sult
0,5 0,7 0,9 1,1 1,3 1,50
0,3
0,6
0,9
1,2
1,5
CC CC
yc
MRL
CC 1,12
CC 1,25
Detection CapabilityCase of a permitted limit (MRL)
2
2
1, .
11
xx
xx
JISytbxay
i
MRLdfMRLc
b
ayCC c
2
2
1, .
11
xx
xCC
JISytbCCay
i
dfc
Detection CapabilityCase of no established permitted limit or banned substance
Xblank CC
Signal orConcentration
CC
≠
+2.33blank +1.64sample
Presence of Heteroscedasticity
Nitroimidazoles in plasma (MNZ-OH)
Residuals plot• “<“ - shape
Plot of S vs conc• Linear relationship
between S and concentration
Heteroscedasticity
Impact on CCa and CCb• CCa and CCb are
incorrectly calculated
• Sblank ↓ CCa ↓
• CCb ↓ or ↑
MNZ-OH y = 0,0831x + 0,0023
R2 = 0,9924
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
0,45
0,5
0 1 2 3 4 5 6
Conc
St
De
v
RESIDUALS PLOT OF THE LINEAR MODEL
-0,8
-0,6
-0,4
-0,2
0
0,2
0,4
0,6
0,8
0 1 2 3 4 5 6
Concentration
Res
idua
l
Other examples
RNZy = 0,0764x - 0,0172
R2 = 0,7899
-0,05
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5
Conc
St
De
v
AMOZy = 0,0426x + 0,0105
R2 = 0,9768
-0,02
0
0,02
0,04
0,06
0,08
0,1
0,12
0,14
-0,5 0 0,5 1 1,5 2 2,5 3
Conc
St
Dev
MNZ y = 0,0389x + 0,0146
R2 = 0,7416
0
0,02
0,04
0,06
0,08
0,1
0,12
-0,5 0 0,5 1 1,5 2 2,5 3
Conc
St
De
v
•Nitroimidazoles in plasma •Nitrofurans in honey
•Corticosteroids in liverTriamcinolone Acetonide y = 0,1975x - 0,1607
R2 = 0,8451
-0,2
-0,1
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5
Conc
St
Dev
Weighted regression equations for CC and CC
2
2
221,
..
1y
wii
w
iCC
df SxxwJ
x
wJS
b
tCC
2
2
22,,
..
1y
wii
w
iCC
df SxxwJ
x
wJS
bCC
•Solved by iteration
Conclusion detection capability
Many definitions of detection limits• detection limit (≈ CC_banned substances)• determination limit (≈ CC_banned substances)• Quantition limit
Complicated statistics KISS
• demonstrate with real (spiked) samples at low concentration level practical limit of detection
Selectivity/Specificity
Identity: Signal to be attributed to the analyte • GLC (change column/polarity), GC/MS, Infra-red
Selectivity: The ability of the method to determine accurately the analyte of interest in the presence of other components in a sample matrix under the stated conditions of the test.
Specificity is a state of perfect selectivity
Selectivity
The procedure to establish selectivity:• Analyze samples and reference materials• Assess the ability of the methods to confirm
identity and measure the analyte• Choose the more appropriate method.• Analyze samples • Examine the effect of interferences
Selectivity: Verification of the identification criteria (2002/657/EC) MS – criteria
• 3 or 4 identification points• 1 precursor and 2 transition ions
• Relative ion intensities LC – criteria
• Relative retention time (RRT): +/- 2.5 % (LC)
UV – criteria• Spectrum match• +/- 3 nm
CC is concentration at or above the calculated CC for which the ID criteria are fulfilled in 95% of the cases.
CC is concentration at or above the calculated CC for which the ID criteria are fulfilled in 50% of the cases.
Ruggedness and Robustness
Intra-laboratory study to check changes due to environmental and/or operating conditions • Usually it is part of method development• Deliberate changes in
• Temperature• Reagents ( e.g. different batches)• Extraction time• Composition in the sample• etc
Precision – ISO 5725 1-6 (1994)
Expresses the closeness of agreement (dispersion Expresses the closeness of agreement (dispersion level, relative standard deviation) between a series of level, relative standard deviation) between a series of measurements from multiple sampling of the same measurements from multiple sampling of the same homogeneous sample (independent assays) under homogeneous sample (independent assays) under prescribed conditions.prescribed conditions.
Irrespective of whether mean is a correct representation of the true value.
Gives information on random errorsrandom errors Evaluated at three levelsthree levels: repeatabilityrepeatability intermediate precision (within laboratory)intermediate precision (within laboratory) reproducibility (between laboratory)reproducibility (between laboratory)
Precision (cont.) – ISO 5725 1-6 (1994)
RepeatabilityRepeatability: precision under conditions where the results of independent assays are obtained by the same analytical procedure, on identical samples, in the same lab, by the same operator, using the same equipment and during short interval of time
Intermediate precisionIntermediate precision: ISO recognizes M-factor M-factor different intermediate precision conditionsdifferent intermediate precision conditions (M = 1, 2 or 3) M = 1M = 1: only 1 of 3 factors (operator, equipment, time) is
differentM= 2 or 3M= 2 or 3: 2 or all 3 factors differ between determinations
Precision (cont.) – ISO 5725 1-6 (1994)
ReproducibilityReproducibility: precision under conditions where results obtained: by same analytical procedureon identical sample in different laboratories, different operators, different
equipment Reproducibility established by interlaboratory study
(standardisationstandardisation of an analytical procedure)
Intermediate precisionIntermediate precision RepeatabilityRepeatability ReproducibilityReproducibility
Evaluation of Precision 10 samples for each conc.under r,R, within lab R
• Standard Deviation
Determination in pairs under r,R, within lab R• Std. Dev. between two single determinations
• a-b, the difference between the values, d, the number of pairs
1
)(1
2
n
xxn
i 100x
sRSD
d
bas ii
2
)( 2
sr
SR
SRw
Repeatability (r) and within-lab reproducibility (Rw) ANOVA table for a single factor balanced design with 3 replicate samples on the same day.
Source Sum of squares df Mean Squares Expected mean squares
day SSdays ndays - 1 MSdays =
SSdays / (ndays – 1)
σrepl² + 3σdays²
replicate SSrepl nT – ndays MSrepl =
SSrepl / (nT – ndays)
σrepl²
Total SST nT – 1 SST = MST / (nT – 1)
repeatability (Sr²) and within-lab reproducibility variances (SRw²)
Sr² = Srepl²
SRw² = Sr² + Sdays²
The Srepl²and Sdays² can be obtained from mean squares as (nrepl = 3):
Srepl² = MSrepl
Sdays² = (MSdays – MSrepl) / 3
Repeatability and reproducibility
The value of 2.8?• Variance of difference between 2 replicate
measurements is 2s² • Confidence interval at 95% level on the difference is
0 ± 1.96 √2 s ± 1.96 x 1.41 sr = ± 2.8 sr
95% probability that difference between duplicate determinations will not exceed 2.8 sr
r = limit of the repeatability r = 2.8 sr
R = limit of the reproducibility R = 2.8 SR
Precision criteria 2002/657/CE
Horwitz: RSDR(%) = 2(1-0.5logC)
-140
-100
-60
-20
20
60
100
1,E
+00
1,E
-01
1,E
-02
1,E
-03
1,E
-04
1,E
-05
1,E
-06
1,E
-07
1,E
-08
1,E
-09
1,E
-10
1,E
-11
1,E
-12
1,E
-13
1,E
-14
C
%
Drugs
Aflatoxines
Dioxins
narcotics in food pesticide residu's
CRRSD log5,012%
Determination of Trueness
Using Certified Reference Materials Using RM or In-house materials Using Reference methods
• Single sample• Many samples
Via Interlaboratory study
Trueness, extraction yield (recovery) and apparent recovery Trueness means the closeness of agreement
between the average value obtained from a large series of test results and an accepted reference value. Trueness is usually expressed as bias
Recovery (extraction yield): yield of a preconcentration or extraction stage of an analytical process for an analyte divided by amount of analyte in the original sample.
Apparent recovery: observed value derived from an analytical procedure by means of a calibration graph divided by reference value.
Trueness criteria 2002/657/CE
When no such CRMs are available, it is acceptable that trueness of measurements is assessed through recovery of additions of known amounts of the analyte(s) to a blank matrix. Data corrected with the mean recovery are only acceptable when they fall within the ranges