A Step in the Right Direction? the development of USP chapter <1210>

Charles Y. Tan, PhDUSP Statistics Expert Committee

OutlineIntroduction of <1210>Key topics

Accuracy and PrecisionLinearityLOD, LOQ, range

Summary

USP <1210>United States PharmacopeiaGeneral Chapters<1210> Statistical Tools for Method

ValidationCurrent status: a draft is published in

Pharmacopeial Forum 40(5) [Sept-Oct 2014]Seek public comments

Purpose of <1210>A companion chapter to <1225> Validation of

Compendial ProceduresUSP <1225> and ICH Q2(R1)USP <1033> Biological Assay Validation

Statistical toolsTOST, statistical equivalenceStatistical power, experimental designtolerance intervals, prediction intervalsRisk assessment, Bayesian analysisAIC for calibration model selection

Recent FrameworkLife cycle perspective

procedure designperformance qualification / validationongoing performance verification

ATP: Analytical Target ProfilePre-specified acceptance criteriaAssume established

Validation: confirmatory stepStatistical interpretation of “validation”

Performance Characteristics Different statistical treatmentsTier 1: accuracy and precision

Statistical “proof” ATP is metEquivalence test / TOSTSample size / power, DOE

Tier 2: linearity, LODRelaxed evidential standard, estimationSample size / power optional

Key topicsUSP General Chapter <1210>Statistical Tools for Method Validation

Accuracy and PrecisionSeparate Assessment Of Accuracy And Precision

Confidence interval within acceptance criteria from ATP

Combined Validation Of Accuracy And Precision γ-expectation tolerance interval: 100γ%

prediction interval for a future observation,Pr (-λ ≤ Y ≤ λ) ≥ γ

γ-content tolerance interval: 100γ% confidence of all future observations

Bayesian tolerance interval

Experimental ConditionYij = μ + Ci + EijCi: experimental condition

combination of ruggedness factors: analyst, equipment, or day

DOE: experience the full domain of operating conditions

As independent as possibleEij: replication within each conditionOne-way analysis (w/ random factor): why?

Separate AssessmentClosed form formulas:

Accuracy: classic confidence interval for biasPrecision: confidence interval for total variability

under one-way layout (Graybill and Wang)Power and sample size calculationStatement of the parameters: bias, variance

Eg. CI of bias: [-0.4%, 1.1%], within ±5% (ATP)Eg. CI of total variability: ≤2.4%, within 3% (ATP)

Implicit risk level: 95% confidence intervals

Combine Accuracy and PrecisionStatement of observation(s)

Closed form formulas, but a bit more complicate 99%-expectation tolerance interval: eg. [-4.3%, 5.0%]

within ±10% (ATP) 99%-content tolerance interval: eg. [-5.9%, 6.6%]

within ±15% (ATP)Bayesian tolerance interval

“the aid of an experienced statistician is recommended”

Simpler Alternative: directly assess the risk with the λ given in ATPPr (-λ ≤ deviation from truth ≤ λ|data)

Scale of AnalysisPooling variances is central to stat analysis

Variance estimates with df=2 are highly unstableNeed to pool across samples, levels

Variance at mass or concentration scale/unitIncrease with level

Solutions: Normalize with constants, eg. Label claim

Normalizing by observed averages makes stat analysis too complicated

Log transformation%NSD and %RSD

LinearityInternal performance characteristic

External view: accuracy and precisionTransparency => credibility

Appropriateness of standard curve fittingA modelA range

Better than the alternatives (all models are approximations)Proportional: model: Y = β1X + εStraight line: Y = β0 + β1X + εQuadratic model: Y = β0 + β1X + β2X2 + ε

Current PracticesPearson correlation coefficient

Anscombe's quartetLack-of-fit F test

independent replicate Mandel’s F-test, the quality coefficient, and the Mark–

Workman testTest of significance

Evidential standard: low since it gives the benefit of doubt to the model you want

Good precision may be “penalized” with a high false rejection rate

Poor precision is “rewarded” with false confirmation of the simpler and more convenient model

Anscombe's Quartet

Two New ProposalsEquivalence test, TOST, in concentration units

Define maximum allowable bias due to calibration in ATPConstruct 90% confidence interval for the bias

comparing the proposed model to a slightly more flexible model

Closed form formula, complexEvidential standard: could be high, depend on allowable

biasAkaike Information Criterion, AICc

Compare the AICc of the proposed model to a slightly more flexible model (smaller wins)

Very simple calculationsEvidential standard: most likely among candidates

Different Burden of ProofHypothesis Testing: Neyman-Pearson

Frame the issue: null versus alternative hypothesesGoal: reject the null hypothesisNull hypothesis: protected regardless of amount of dataDecision standard: beyond reasonable doubtLegal analogy: criminal trial

Information Criteria: Kullback-LeiblerFrame the issue: a set of candidate modelsGoal: find the best approximation to the truthBest: most parsimonious model given the data at handDecision standard: most likely among candidatesLegal analogy: civil trial

Stepping-stone or tactical questions: information criteria are apt alternatives to hypothesis tests

IUPAC/ISO LOD (RC and RD)

IUPAC/ISO LOD

LOD: Using Prediction Bounds

Range and LOQRange

suitable level of precision and accuracyBoth upper and lower limits

LOQ (LLOQ)acceptable precision and accuracylower limit

LOQ versus LODOnly one is needed for each useLOQ for quantitative tests LOD for qualitative limit tests

LOQ calculation in ICH Q2: candidate starting values

SummaryA draft of USP <1210> is published, seeking

public commentsA step in the right direction?

More than a bag of toolsImplement modern validation concepts with a

statistical structuralMore tools development neededMore statisticians involvement needed in

pharmacopeia and ICH development