Midwest Biopharmaceutical Statistics Workshop Muncie IN, May 24-26, 2010 Statistical Considerations...

Midwest Biopharmaceutical Statistics WorkshopMuncie IN, May 24-26, 2010

Statistical Considerations for Defining Cut Points and Titers in

Anti-Drug Antibody (ADA) Assays

Ken Goldberg, Non-Clinical Statistics

Johnson & Johnson Pharmaceutical Research & Development, LLC, Chesterbrook, PA

Outline

• Introduction– Why are ADA and IR assays important?

• Two case studies1. RIA: How to define %binding?

2.ECL: How to define titer cut point?

3.Both use a Huber 3-parameter nonlinear logistic regression

Statistical Considerations for Defining Cut Points and Titers in ADA Assays.Ken Goldberg. Midwest Biopharmaceutical Statistics Workshop, May 24-26, 2010. Slide 2

Immune Response (IR) Assay

• Primary question: ADA, Yes or No?

• Every biologic must be evaluated.

• Safety and Efficacy concerns.

• Too much IR can kill a compound.


Biological Drug Products are Different than Traditional

Small Molecule Drugs

• Made by cells not chemists

• Complicated manufacturing process

• Small & simple vs large & complex chemical structures


Reference: Genentech, Inc. http://www.gene.com/gene/about/views/followon-biologics.html


Adverse Clinical Sequelae

• Hypersensitivity & autoimmunity

• Altered PK– Drug neutralization– Abnormal biodistribution– Enhanced clearance rate

Regulatory bodies require ADA

evaluation for all biologics


Immune Response (IR) Assay Challenges

• Cut Point for confidence that screening bioassay response (eg, ECL, OD, RLU, CPM) reflects immunogenicity

• Statistical issues of variance components, distributions, outliers, …


Screening Cut Point Flags 5% of Naïve Samples as False

Positive• Use Mean + 1.645 x SD with caution

– Only for normally independently distributed data without outliers

– Usually requires at least a transformation like logs

• Nonparametric often easier– Simply use 95th percentile– Caution if unbalanced design


ELISA Activity

Positive Control

Negative Control

PatientA

PatientB

PatientC

AssayControl

1.689 0.153 0.055 0.412 1.999 0.123


-1/ OD̂ .75

Frequency

0.0-3.5-7.0-10.5-14.0-17.5-21.0-24.5

25

20

15

10

5

0

-1.61Mean -3.872StDev 1.381N 118

Histogram of -1/ OD^.75Normal Distribution Overlaid

Mean and Standard Deviation based on mixed effects analysis of 117 non-outliers.


ELISA Cut Point Example

Analysis of an RIA Cut Point Assay Validation Experiment

• 6 Assay controls

• 2 Analysts with 3 assays each

• 2 Populations (Normal and Diabetes)

• 75 Naïve Human Serum samples

• Nonnormal data

• Unequal variances


RIA Histogram of 450 Naïve Sample Results

Transformed %Binding = ln(35+%Binding). Parametric Cut Point = 10.757 ± 3.524.Transformed Cut Point = 3.823 based on adjusted mean = 3.402 and total standard deviation = 0.256.


RIA Normal Probability Plot of 450 Naïve Sample Results

Transformed %Binding = ln(35+%Binding). Parametric Cut Point = 10.757 ± 3.524.Transformed Cut Point = 3.823 based on adjusted mean = 3.402 and total standard deviation = 0.256.


SAS Code


proc mixed; * For Cut Point; class sample run analyst; model t35Pct0_100= / ddfm=sat; random sample; random sample / type=sp(exp)(tube) subject=analyst*run; repeated / group=analyst*run;

proc mixed; * For Example Hypothesis Test; class sample run analyst; model t35Pct0_100 = Analyst Tube / ddfm=sat; random sample; random intercept tube / type=fa0(2) subject=analyst*run; repeated / group=analyst*run;

My RIA Notation


MinCPM = Minimum of the 2 Sample CPMsMaxCPM = Maximum of the 2 Sample CPMsAvgCPM = Average of the 2 Sample CPMsCV = Coefficient of Variation of the 2 Sample CPMs

B0 = Average of all 6 “Validation sample 0 ng/mL” CPMsB100 = Average of all 6 “Validation sample 100 ng/mL” CPMsB250 = Average of all 6 “Validation sample 250 ng/mL” CPMsB1000 = Average of all 6 “Validation sample 1000 ng/mL” CPMsNSB = Average of all 2-6 “NSB” (Non-Specific Binding) CPMsTC = Average of all 2-6 “TC” (Total Count) CPMs

Some RIA %Binding Definitions


Response

%CV Limit

Sample N

Sample Mean

Sample SD

Addend1

Sample %CV1

(MinCPM-B0)/(B100-B0)*100 450 -3.490 7.968 65 13.0

(AvgCPM-B0)/(B100-B0)*100 25 420 -1.173 8.373 85 10.0

(MinCPM-NSB)/(TC-NSB)*100 450 1.249 0.841 4.4 14.9

MinCPM/NSB 450 1.321 0.218 -0.7 35.0

AvgCPM/(TC-NSB)*100 20 403 5.459 1.119 3 13.2

MinCPM-B0 450 -59.339 151.356 1000 16.1

MinCPM/sqrt(B100*B0) 450 0.549 0.084 0 15.3

1CV of (Response + Addend) = Standard Deviation / (Mean + Addend) x 100%.

Addend chosen so that CV is not related to control concentration.

How to Choose the RIA %Binding Definition?


New versus Old RIA%Binding Definitions

• New: (MinCPM – B0) / (B100 – B0) – Repeat if CV > 25% and (MaxCPM – B0) /

(B100 – B0) > 12.0% (the Cut Point)

• Old: (AvgCPM – NSB) / TC– Repeat if CV > 20%


Attributes of Selected RIA %Binding Definitions


Response %CV Limit

Cut Point

LOD

(ng/mL) 0 ng/mL %Pos.

N

(MinCPM-B0)/(B100-B0) .120 23.5 0.04 450

(AvgCPM-B0)/(B100-B0) 25 .149 25.5 1.29 420

(AvgCPM-B0)/(B100-B0) 20 .153 25.0 0.10 403

(AvgCPM-NSB)/TC 20 3.380 31.7 0.112 403

RIA Validation Control Curve with Lower 1-sided 95% Prediction Limit 65 + %Binding = A+B·ConcentrationC


A Logistic Curve with an Infinite Plateau is Linear wrt X

C + R XH / ( MH + XH) =

Substitute α = C, = H, and R/β = MH

α + R X / (R/β + X) =Multiply second term by β/β

α + β R X / ( R + βX)Apply L’Hopital’s rule

Lim[ α + R β X / (R + β X) ] = α + β X (R)


RIA Naïve Sample %Binding vs Test Tube Order by Population


180160140120100806040200

40

30

20

10

0

-10

-20

-30

Tubepair

Min

Pct

0_100

DiabetesNormal

Population

Scatterplot of MinPct0_100 vs Tubepair

RIA Naïve Sample %Binding vs Test Tube Order by Analyst


180160140120100806040200

40

30

20

10

0

-10

-20

-30

Tubepair

Min

Pct

0_100

12.05

12

Analyst

Scatterplot of MinPct0_100 vs Tubepair

RIA Naïve Sample %Binding vs Test Tube Order by Analyst and Run


15010050

400

300

200

15010050

400

300

20015010050

1, 1

Tubepair

ln(3

5+

Pct

0_100)*

100

385.1

1, 2 1, 3

2, 1 2, 2 2, 3

385.1

Scatterplot of ln(35+Pct0_100)*100 vs Tubepair

Panel variables: Analyst, Run

RIA Naïve Sample Means vs Test Tube Order by Population, Analyst and Run


15010050

450

400

350

300

250

15010050

450

400

350

300

250

15010050

1, 1

Tubepair

ln(M

eanPct

+35)*

100

1, 2 1, 3

2, 1 2, 2 2, 3

DiabetesNormal

Population

Scatterplot of ln(MeanPct+35)*100 vs Tubepair


RIA Naïve Sample Mean %Binding vs CV by Analyst and Run


40-4

450

400

350

300

250

40-4

450

400

350

300

250

40-4

1, 1

lnCV

ln(M

eanPct

+35)*

100

1, 2 1, 3

2, 1 2, 2 2, 3

Scatterplot of ln(MeanPct+35)*100 vs lnCV


RIA Naïve Sample Minimum %Binding vs CV by Analyst and Run


40-4

400

300

200

40-4

400

300

20040-4

1, 1

lnCV

ln(3

5+

Pct

0_100)*

100

385.1

1, 2 1, 3

2, 1 2, 2 2, 3

385.1

Scatterplot of ln(35+Pct0_100)*100 vs lnCV


RIA Naïve Sample CPM CV vs Mean by Analyst


RIA Naïve Sample CPM CV vs Mean by Population and Control


RIA Probability Plots of ln(35+%Binding)•100 by Analyst


425400375350325300275250

99.9

99

95

90

80706050403020

10

5

1

0.1

ln(35+Pct0_100)*100

Perc

ent

385.1

344.0 24.90 225 2.052 <0.005339.9 24.84 225 3.863 <0.005

Mean StDev N AD P

12

Analyst

Probability Plot of ln(35+Pct0_100)*100

RIA Probability Plots of ln(35+%Binding)•100 by Population


450400350300250

99.9

99

95

90

80706050403020

10

5

1

0.1

ln(35+Pct0_100)*100

Perc

ent

341.8 19.87 150 0.542 0.162342.0 27.14 300 1.573 <0.005

Mean StDev N AD P

DiabetesNormal

Population

Probability Plot of ln(35+Pct0_100)*100Normal

Electrochemiluminescence (ECL) BioVeris Assay

• New way to determine screening cut point (Data = naïve samples)

• New way to determine titer cut point (not equal to screening cut point) (Data = positive samples’ Titration series)

• Estimator of Titer within-assay CV


Screening Cut Point DeterminationECL of Naïve Sample vs Diluent Alone with Cutoffs by Diluent ECL


Titer Definition

• Smallest distinct dilution in a titration series with a negative response– Response is Sample ECL mean / Diluent

Control ECL mean in this case study


Plot where Sample/Diluent Control ECL Ratio < 4 for 1 Selected Plate out of 24


Potential Problems with a Common Screening and

Titer Cut Point

• Highly diluted samples tend to be positive!– The opposite would not be a problem

• Titration curve too flat at cut point– Makes the titer highly variable– Common


Titer Cut Point Defined

• The continuous titer inverse predicted from it has CV ≤ 30.0% with 95% confidence

– 30.0% makes best case CV = worst case CV in ideal assay

– Continuous titer is exact dilution giving cut point (only as a theoretical concept)


Asymptotic CV

• CV Standard deviation of natural log ratio or titer

• CV of dilution@ratio CV of ratio / slope of titration curve@ratio

• CV of dilution decreases as ratio and slope increase

• These CVs are within-plate CVs


Four Theoretical Titer Distributions


42

50

25

0

Discrete Titer

Per

cent

5050 5050 5050

CV = 34.7% = ln(F)/ 250% at X and 50% at X*F. CV=ln(F)/2

842

75

50

25

0

Discrete Titer

Per

cent

12.5

75

12.5

CV = 34.7% = ln(F)/ 275% at X, 12.5% each at X/F and X*F

8421

50

25

0

Discrete Titer

Per

cent

1

4949

1

30% CV of Continuous Titer37.5%=> Discrete Titer CV =

168421

75

50

25

0

Discrete Titer

Per

cent

0.0312.39

75.16

12.390.03

30% CV of Continuous Titer=> Discrete Titer CV = 34.7%

Titer Cut Point Defined• A continuous (interpolated) titer inverse

predicted from it has CV<30.0% with 95% confidence– Exact dilution giving cut point (eg, 1.357

ratio) is the continuous titer– Continuous titer used here only as a

theoretical concept– Our cut-point 5 SD above diluent mean so

false-positives of noncensored titers unlikely


Summary

• All biologics need ADA evaluation

• Use controls to adjust for plate-to-plate variance and minimize the LOD

• Define titer cut point so best case CV = worst case CV in ideal assay


Acknowledgements:• Sheng Dai• Allen Schantz

Reference: Shankar, G. et al, (2008). Recommendations for the

validation of immunoassays used for detection of host antibodies against biotechnology products. Journal of Pharmaceutical and Biomedical Analysis. 48:1267–1281.

• Pam Cawood• Gopi Shankar• Bill Pikounis


Midwest Biopharmaceutical Statistics Workshop Muncie IN, May 24-26, 2010 Statistical Considerations...

Documents

Transcript of Midwest Biopharmaceutical Statistics Workshop Muncie IN, May 24-26, 2010 Statistical Considerations...