Midwest Biopharmaceutical Statistics Workshop Muncie IN, May 24-26, 2010 Statistical Considerations...

Midwest Biopharmaceutical Statistics WorkshopMuncie IN, May 24-26, 2010

Statistical Considerations for Defining Cut Points and Titers in

Anti-Drug Antibody (ADA) Assays

Ken Goldberg, Non-Clinical Statistics

Johnson & Johnson Pharmaceutical Research & Development, LLC, Chesterbrook, PA

Outline

• Introduction– Why are ADA and IR assays important?

• Two case studies1. RIA: How to define %binding?

2.ECL: How to define titer cut point?

3.Both use a Huber 3-parameter nonlinear logistic regression

Statistical Considerations for Defining Cut Points and Titers in ADA Assays.Ken Goldberg. Midwest Biopharmaceutical Statistics Workshop, May 24-26, 2010. Slide 2

Immune Response (IR) Assay

• Primary question: ADA, Yes or No?

• Every biologic must be evaluated.

• Safety and Efficacy concerns.

• Too much IR can kill a compound.

Biological Drug Products are Different than Traditional

Small Molecule Drugs

• Made by cells not chemists

• Complicated manufacturing process

• Small & simple vs large & complex chemical structures

Reference: Genentech, Inc. http://www.gene.com/gene/about/views/followon-biologics.html

Adverse Clinical Sequelae

• Hypersensitivity & autoimmunity

• Altered PK– Drug neutralization– Abnormal biodistribution– Enhanced clearance rate

Regulatory bodies require ADA

evaluation for all biologics

Immune Response (IR) Assay Challenges

• Cut Point for confidence that screening bioassay response (eg, ECL, OD, RLU, CPM) reflects immunogenicity

• Statistical issues of variance components, distributions, outliers, …

Screening Cut Point Flags 5% of Naïve Samples as False

Positive• Use Mean + 1.645 x SD with caution

– Only for normally independently distributed data without outliers

– Usually requires at least a transformation like logs

• Nonparametric often easier– Simply use 95th percentile– Caution if unbalanced design

ELISA Activity

Positive Control

Negative Control

PatientA

PatientB

PatientC

AssayControl

1.689 0.153 0.055 0.412 1.999 0.123

-1/ OD̂ .75

Frequency

0.0-3.5-7.0-10.5-14.0-17.5-21.0-24.5

-1.61Mean -3.872StDev 1.381N 118

Histogram of -1/ OD^.75Normal Distribution Overlaid

Mean and Standard Deviation based on mixed effects analysis of 117 non-outliers.

ELISA Cut Point Example

Analysis of an RIA Cut Point Assay Validation Experiment

• 6 Assay controls

• 2 Analysts with 3 assays each

• 2 Populations (Normal and Diabetes)

• 75 Naïve Human Serum samples

• Nonnormal data

• Unequal variances

RIA Histogram of 450 Naïve Sample Results

Transformed %Binding = ln(35+%Binding). Parametric Cut Point = 10.757 ± 3.524.Transformed Cut Point = 3.823 based on adjusted mean = 3.402 and total standard deviation = 0.256.

RIA Normal Probability Plot of 450 Naïve Sample Results

Transformed %Binding = ln(35+%Binding). Parametric Cut Point = 10.757 ± 3.524.Transformed Cut Point = 3.823 based on adjusted mean = 3.402 and total standard deviation = 0.256.

SAS Code

proc mixed; * For Cut Point; class sample run analyst; model t35Pct0_100= / ddfm=sat; random sample; random sample / type=sp(exp)(tube) subject=analyst*run; repeated / group=analyst*run;

proc mixed; * For Example Hypothesis Test; class sample run analyst; model t35Pct0_100 = Analyst Tube / ddfm=sat; random sample; random intercept tube / type=fa0(2) subject=analyst*run; repeated / group=analyst*run;

My RIA Notation

MinCPM = Minimum of the 2 Sample CPMsMaxCPM = Maximum of the 2 Sample CPMsAvgCPM = Average of the 2 Sample CPMsCV = Coefficient of Variation of the 2 Sample CPMs

B0 = Average of all 6 “Validation sample 0 ng/mL” CPMsB100 = Average of all 6 “Validation sample 100 ng/mL” CPMsB250 = Average of all 6 “Validation sample 250 ng/mL” CPMsB1000 = Average of all 6 “Validation sample 1000 ng/mL” CPMsNSB = Average of all 2-6 “NSB” (Non-Specific Binding) CPMsTC = Average of all 2-6 “TC” (Total Count) CPMs

Some RIA %Binding Definitions

Response

%CV Limit

Sample N

Sample Mean

Sample SD

Addend1

Sample %CV1

(MinCPM-B0)/(B100-B0)*100 450 -3.490 7.968 65 13.0

(AvgCPM-B0)/(B100-B0)*100 25 420 -1.173 8.373 85 10.0

(MinCPM-NSB)/(TC-NSB)*100 450 1.249 0.841 4.4 14.9

MinCPM/NSB 450 1.321 0.218 -0.7 35.0

AvgCPM/(TC-NSB)*100 20 403 5.459 1.119 3 13.2

MinCPM-B0 450 -59.339 151.356 1000 16.1

MinCPM/sqrt(B100*B0) 450 0.549 0.084 0 15.3

1CV of (Response + Addend) = Standard Deviation / (Mean + Addend) x 100%.

Addend chosen so that CV is not related to control concentration.

How to Choose the RIA %Binding Definition?

New versus Old RIA%Binding Definitions

• New: (MinCPM – B0) / (B100 – B0) – Repeat if CV > 25% and (MaxCPM – B0) /

(B100 – B0) > 12.0% (the Cut Point)

• Old: (AvgCPM – NSB) / TC– Repeat if CV > 20%

Attributes of Selected RIA %Binding Definitions

Response %CV Limit

Cut Point

(ng/mL) 0 ng/mL %Pos.

(MinCPM-B0)/(B100-B0) .120 23.5 0.04 450

(AvgCPM-B0)/(B100-B0) 25 .149 25.5 1.29 420

(AvgCPM-B0)/(B100-B0) 20 .153 25.0 0.10 403

(AvgCPM-NSB)/TC 20 3.380 31.7 0.112 403

RIA Validation Control Curve with Lower 1-sided 95% Prediction Limit 65 + %Binding = A+B·ConcentrationC

A Logistic Curve with an Infinite Plateau is Linear wrt X

C + R XH / ( MH + XH) =

Substitute α = C, = H, and R/β = MH

α + R X / (R/β + X) =Multiply second term by β/β

α + β R X / ( R + βX)Apply L’Hopital’s rule

Lim[ α + R β X / (R + β X) ] = α + β X (R)

RIA Naïve Sample %Binding vs Test Tube Order by Population

180160140120100806040200

Tubepair

DiabetesNormal

Population

Scatterplot of MinPct0_100 vs Tubepair

RIA Naïve Sample %Binding vs Test Tube Order by Analyst

180160140120100806040200

Tubepair

Analyst

Scatterplot of MinPct0_100 vs Tubepair

RIA Naïve Sample %Binding vs Test Tube Order by Analyst and Run

15010050

20015010050

Tubepair

0_100)*

1, 2 1, 3

2, 1 2, 2 2, 3

Scatterplot of ln(35+Pct0_100)*100 vs Tubepair

Panel variables: Analyst, Run

RIA Naïve Sample Means vs Test Tube Order by Population, Analyst and Run

15010050

Tubepair

eanPct

1, 2 1, 3

2, 1 2, 2 2, 3

DiabetesNormal

Population

Scatterplot of ln(MeanPct+35)*100 vs Tubepair

RIA Naïve Sample Mean %Binding vs CV by Analyst and Run

eanPct

1, 2 1, 3

2, 1 2, 2 2, 3

Scatterplot of ln(MeanPct+35)*100 vs lnCV

RIA Naïve Sample Minimum %Binding vs CV by Analyst and Run

20040-4

0_100)*

1, 2 1, 3

2, 1 2, 2 2, 3

Scatterplot of ln(35+Pct0_100)*100 vs lnCV

RIA Naïve Sample CPM CV vs Mean by Analyst

RIA Naïve Sample CPM CV vs Mean by Population and Control

RIA Probability Plots of ln(35+%Binding)•100 by Analyst

425400375350325300275250

80706050403020

ln(35+Pct0_100)*100

344.0 24.90 225 2.052 <0.005339.9 24.84 225 3.863 <0.005

Mean StDev N AD P

Analyst

Probability Plot of ln(35+Pct0_100)*100

RIA Probability Plots of ln(35+%Binding)•100 by Population

450400350300250

80706050403020

ln(35+Pct0_100)*100

341.8 19.87 150 0.542 0.162342.0 27.14 300 1.573 <0.005

Mean StDev N AD P

DiabetesNormal

Population

Probability Plot of ln(35+Pct0_100)*100Normal

Electrochemiluminescence (ECL) BioVeris Assay

• New way to determine screening cut point (Data = naïve samples)

• New way to determine titer cut point (not equal to screening cut point) (Data = positive samples’ Titration series)

• Estimator of Titer within-assay CV

Screening Cut Point DeterminationECL of Naïve Sample vs Diluent Alone with Cutoffs by Diluent ECL

Titer Definition

• Smallest distinct dilution in a titration series with a negative response– Response is Sample ECL mean / Diluent

Control ECL mean in this case study

Plot where Sample/Diluent Control ECL Ratio < 4 for 1 Selected Plate out of 24

Potential Problems with a Common Screening and

Titer Cut Point

• Highly diluted samples tend to be positive!– The opposite would not be a problem

• Titration curve too flat at cut point– Makes the titer highly variable– Common

Titer Cut Point Defined

• The continuous titer inverse predicted from it has CV ≤ 30.0% with 95% confidence

– 30.0% makes best case CV = worst case CV in ideal assay

– Continuous titer is exact dilution giving cut point (only as a theoretical concept)

Asymptotic CV

• CV Standard deviation of natural log ratio or titer

• CV of dilution@ratio CV of ratio / slope of titration curve@ratio

• CV of dilution decreases as ratio and slope increase

• These CVs are within-plate CVs

Four Theoretical Titer Distributions

Discrete Titer

5050 5050 5050

CV = 34.7% = ln(F)/ 250% at X and 50% at X*F. CV=ln(F)/2

Discrete Titer

CV = 34.7% = ln(F)/ 275% at X, 12.5% each at X/F and X*F

Discrete Titer

30% CV of Continuous Titer37.5%=> Discrete Titer CV =

168421

Discrete Titer

0.0312.39

12.390.03

30% CV of Continuous Titer=> Discrete Titer CV = 34.7%

Titer Cut Point Defined• A continuous (interpolated) titer inverse

predicted from it has CV<30.0% with 95% confidence– Exact dilution giving cut point (eg, 1.357

ratio) is the continuous titer– Continuous titer used here only as a

theoretical concept– Our cut-point 5 SD above diluent mean so

false-positives of noncensored titers unlikely

Summary

• All biologics need ADA evaluation

• Use controls to adjust for plate-to-plate variance and minimize the LOD

• Define titer cut point so best case CV = worst case CV in ideal assay

Acknowledgements:• Sheng Dai• Allen Schantz

Reference: Shankar, G. et al, (2008). Recommendations for the

validation of immunoassays used for detection of host antibodies against biotechnology products. Journal of Pharmaceutical and Biomedical Analysis. 48:1267–1281.

• Pam Cawood• Gopi Shankar• Bill Pikounis

Midwest Biopharmaceutical Statistics Workshop Muncie IN, May 24-26, 2010 Statistical Considerations...

Documents

Transcript of Midwest Biopharmaceutical Statistics Workshop Muncie IN, May 24-26, 2010 Statistical Considerations...

Biopharmaceutical Failures

Ecdysteroid Titers in Mated and Unmated Drosophila ...

Biopharmaceutical Industry Speakers List - Home Page | …phrma-docs.phrma.org/.../biopharmaceutical-industry-speakers-list... · The Biopharmaceutical Industry Speakers List contains

MUNCIE Governing Young People

Great Deals Muncie Aug 09

Neutralizing antibody titers six months after Comirnaty ...

Native Americans in Muncie (pdf)

Muncie Aviation Q3 Newsletter

MUNCIE COMMUNITY SCHOOLS UPDATE - Indiana. Muncie Community Schools Presentat… · 2016-17 MCS A-F School Grades Storer Elementary School* A Muncie Community School Corporation C

FA6B - Muncie Power Products

HEALTH AND WELLBEING TASKFORCE MUNCIE BEFORE FIVE/MUNCIE ACTION PLAN.

HYDRAULIC FILTERS - Muncie Power Products

A NEW ERA - Muncie Power

24 Hours in Muncie 3

HYDRAULIC RESERVOIRS - Muncie Power Products · WARRANTY INFORMATION The Muncie polyethylene hydraulic oil reservoir, when used with the complete Muncie tractor wet line kit and hoses,

Great Deals Muncie Dec 09

Muncie Great Deals

Muncie Fiscal Situation

Muncie Magazine

24 Hours in Muncie 2