Diagnostic performance measures of ELISA and quantitative PCR tests for porcine circovirus type 2...

Preventive Veterinary Medicine 101 (2011) 79 88

Contents lists available at ScienceDirect

Preventive Veterinary Medicine

j ourna l ho me pag e: ww w.elsev i er .com/ locate /prev etmed

Diagno A anfor por ng Banalysi

Charles H iranJuan Abrahanteb, Michael P. Murtaughb

a Centers for Epidemiology and Animal Health, USDAAPHISVS, 2150 Centre Ave., Bldg. B, MS 2E7 Fort Collins, CO, United Statesb Department of Veterinary and Biomedical Sciences, University of Minnesota, St. Paul, MN, United States

a r t i c l

Article history:Received 16 AReceived in reAccepted 3 Ma

Keywords:BayesianConditional deLatent dataSensitivitySpecicitySwine

1. Introdu

Porcine nonenvelopPCV1 was of the conet al., 1982

CorresponE-mail add

0167-5877/$ doi:10.1016/j. e i n f o

pril 2010vised form 13 April 2011y 2011

pendence

a b s t r a c t

Porcine circovirus 2 (PCV2) is believed to be a necessary but not sufcient underlying causeof porcine circovirus associated disease (PCVAD) in swine (Opriessnig et al., 2007). Sincethe potential threat of PCVAD is dependent on the prevalence of PCV2 in swine populations,accurate diagnostic tests are important for epidemiologic surveillance. Therefore, we eval-uated the diagnostic sensitivity (Se) and specicity (Sp) of a new indirect ELISA and twoquantitative PCR tests for PCV2 in a series of latent class models that used Bayesian esti-mation procedures. A total of 4140 samples from nisher pigs were tested for evidence ofPCV2 by the ELISA and a TaqMan (TM) quantitative PCR, 995 by the ELISA and a SYBR Green(SG) dye-binding PCR, 998 by both PCRs and 993 by all three tests. Overall, the median (95%probability interval) ELISA Se and Sp was 0.85 (0.830.87) and 0.74 (0.680.79), respec-tively, when all three tests were analyzed together at an ELISA absorbance (optical densityor OD) cutoff of 0.3. The TM PCR Se and Sp was 0.86 (0.840.88) and 0.94 (0.870.97),respectively, and the SG PCR Se and Sp was 0.83 (0.810.85) and 0.98 (0.941.00), respec-tively when all three tests were analyzed together at an ELISA OD cutoff of 0.3. Sensitivityanalysis revealed that Sp estimates in general had less stability than Se estimates, but theSG PCRSp was the most stable. Limited conditional dependence between the two PCR testswas detected. We conclude that the ELISA had the highest diagnostic Se at an absorbancecutoff of 0.3, while the SG PCR had the highest diagnostic Sp. The prevalence levels forexposure to PCV2 in nishing swine populations across all analyses ranged from 58 to 100%.

Published by Elsevier B.V.

ction

circoviruses (PCV) PCV1 and PCV2 are small,ed viruses with a single-stranded DNA genome.discovered as a nonpathogenic contaminanttinuous PK-15 cell line in 1975 (Tischer). PCV1 is incorrectly believed to be ubiq-

ding author. Tel.: +1 970 494 7216; fax: +1 970 494 7228.ress: [email protected] (C. Haley).

uitous in United States swine populations (Hines andLukert, 1995; Puvanendiran et al., 2011). PCV2 is rec-ognized as a necessary, but not sufcient, agent ofporcine-circovirus-associated diseases (PCVAD), includingpostweaning multisystemic wasting syndrome (PMWS)and porcine dermatitis and nephropathy syndrome (PDNS)(Harding, 2004; Segals et al., 2005; Turner et al., 2009).Accurate antibody and nucleic acid detection and quan-titation tools are needed to assess and monitor PCV2 incommercial swine populations to better understand thethreat of PCVAD and the impact of disease prevention

see front matter. Published by Elsevier B.V.prevetmed.2011.05.001stic performance measures of ELIScine circovirus type 2 exposure usis

aleya,, Bruce Wagnera, Sumathy Puvanendd quantitative PCR testsayesian latent class

b,

80 C. Haley et al. / Preventive Veterinary Medicine 101 (2011) 79 88

strategies on PCV2 prevalence. The impact of this disease onthe pork industry initiated increased levels of research intoPCVAD pathology and into understanding the connectionbetween PCV2 and PCVAD in terms of immune response.

Antibodserum and ety of substEllis, 2000;However, thof these tesswine popuMcIntosh etmates of PClacking for

Diagnosby using cogold standa1997). Whemance meaa statisticalpositive anestimated vas true PCV

Latent crst is thatmore subposure. The seconstant acease or expthird is thadent given the probabis the samethe other. Blatent classunknown pmaximum lbutions foret al., 2005)

The objetic performPCV2 and ethe U.S. swithe introduPCVAD. Theunderstandof the U.S. sof vaccinatInformationquestions ractions witand respira

2. Materia

2.1. Data a

Questionas part of t(NAHMS) Swith 100 o

for the study based on numbers of hogs and pigs in andthe number of operations with 100 or more hogs in eachstate. This information was obtained from the NationalAgricultural Statistics Service (NASS) Quarterly Hogs and

reporPigs//2bers ofrms, alib.coLandIn

with used004 t

e inven94.2% ations

on thewine 2stratited froate anabilityxcepti

(univ eligibtionna

initia enumventoTwo frst andd by Vs). As

growe taken2 weeles. Wdence assume requod same sampmberace inhethecoplasderal

samn pens

the crutainelot act5 F) fC (39ithin 2res Naes, IA

n off, aody anry at t

were py assays for the presence of PCV2 antibodies inPCR techniques to detect PCV2 DNA in a vari-rates have been described previously (Allan and

Grierson et al., 2004; Opriessnig et al., 2007).e diagnostic sensitivity (Se) and specicity (Sp)

ts often is not well characterized in commerciallations (Olvera et al., 2004; Segals et al., 2005;

al., 2009). In addition, basic epidemiologic esti-V2 seroprevalence and viral DNA prevalence arethe U.S. swine herd.tic performance (Se and Sp) can be determinednrmatory tests as a gold standard, but truerd tests usually do not exist (Saah and Hoover,n there is no gold standard, diagnostic perfor-sures can be estimated by latent class analysis,

technique that utilizes observed data such asd negative PCV2 ELISA or PCR test results andalues of an unobserved, or latent, variable such2 exposure prevalence.lass analysis involves three assumptions. The

a target population should consist of two orpulations with different prevalences of expo-cond is that the accuracy of both tests should beross subpopulations (e.g., regardless of the dis-osure prevalence in the subpopulations). Thet the tests should be conditionally indepen-the disease or exposure status, meaning thatility of a negative or positive result on one test

regardless of a negative or positive test onayesian estimation procedures are a method of

analysis that combine prior knowledge aboutarameters and accompanying data through aikelihood procedure to produce updated distri-

parameters of interest (Ene et al., 2000; Toft.ctives of this study were to assess the diagnos-ance of a new ELISA and two new PCR tests forstimate the prevalence of exposure to PCV2 inne population. The study was initiated prior toction of vaccines for control and prevention of

knowledge from this study will enable bettering of the immunological and virological statuswine herd for PCV2, and facilitate assessmention and control strategies on PCV2 in swine.

on PCV2 exposure status will help addresselated to PCVAD pathogenesis and PCV2 inter-h other diseases, such as porcine reproductivetory syndrome (PRRS).

ls and methods

nd sample collection

naire data and blood samples were collectedhe National Animal Health Monitoring Systemwine 2006 study which focused on operationsr more hogs. Seventeen states were selected

Pigs Hogsnumin FamannFarmtionswerelate 2swinand Operwerethe S

A selecon stprobthe escopewerequesNASSNASSon inpigs. the istere(VMOwithto be(203sampconmal, ThesA bloof thID, nuair spand wor My

Febloodpigs ifroma Vacthe c(687to 4

ice wcultuin Amdrawantiboratotestst (http://usda.mannlib.cornell.edu/usda/nass/000s/2004/HogsPigs-12-28-2004.pdf) for

hogs and pigs in each state. The Farms, Landnd Livestock Operations report (http://usda.rnell.edu/usda/nass/FarmLandIn//2000s/2005/-01-31-2005.pdf) supplied number of opera-100 or more hogs in each state. Both reports

to purposely select 17 states for study. Inhese 17 states accounted for 94.0% of the U.S.tory for operations with more than 100 hogs,

of U.S. operations with more than 100 hogs. with 100 or more hogs from the 17 states that

NASS list frame were eligible to participate in006 study.ed random sample of 5006 operations wasm the NASS list frame. Stratication was basedd herd size, and larger operations had a higher

of being selected. All selected operations, withon of operations that were considered out ofersity farms, research facilities, and prisons),le for an on-site interview with up to threeires. A total of 2230 producers completed thel swine farm questionnaire, administered byerators. The questionnaire included questionsry and housing for all management stages ofollowup questionnaires (514 respondents for

435 respondents for the second) were admin-eterinary Services Veterinary Medical Ofcers

part of the study, 185 producers in 16 statesr/nisher pigs agreed to allow blood samples

from their pigs. Grower/nisher market pigsks of age) were the target population for theseithin-herd sampling was designed to give 95%of detecting at least one PCV2 test-positive ani-ing a within-herd prevalence of at least 10%.irements were met with 35 samples per farm.ple collection form was lled out at the timele draw and included facility/building ID, pen

of pigs in pen, number of pigs sharing the same the sampled area, pig age in weeks, pig gender,r the pigs were vaccinated for PRRS, inuenza,ma.VMOs or Animal Health Technicians obtainedples from randomly selected grower/nisher

on each site. Up to 13 ml of blood was collectedanial vena cava or jugular vein and injected intor tube. Tubes were inverted ve times to mixivator, and allowed to sit vertically at 2024 Cor a minimum of 30 min. Samples were cooledF) immediately after clotting and shipped on4 h of collection to the U.S. Department of Agri-tional Veterinary Services Laboratories (NVSL). At NVSL, tubes were centrifuged and the seraliquoted, and frozen. Aliquots destined for PCV2d DNA testing were sent to the Murtaugh Lab-he University of Minnesota, St. Paul, MN. Threeerformed: an Indirect ELISA to PCV2 antibodies,

C. Haley et al. / Preventive Veterinary Medicine 101 (2011) 79 88 81

a TaqMan (TM) quantitative PCR to PCV2 DNA and a SYBRGreen (SG) quantitative PCR to PCV2 DNA as described inPuvanendiran et al. (2011).

2.2. Statisti

2.2.1. DiagnAn Exce

rect ELISA, TLaboratory presented areported asence in Ct bproduced intioned in PSG PCR resued in the amelting pothrough bloExcel databsion 9.2) ansample collquestionnationnaire dtwo differenitive from nresult werebetween duitive consisremained a

2.2.2. BayesPrior to

of PCV2 poprevalence els of expoof our knowmates on thDNA, nor faexposure topopulation ative test rewhich repoherds that rprior to thethis pattern(USDA, 200eling, the tcross tabulwith >2000tory on Junchi square atabulationssured by thbe proper t

2.2.3. Bayesparameters

Prior esMurtaugh Lmodes wer

ucdavis .edu/diagnostictests/betabuster.html) to get thealpha and beta parameters of the prior beta distributions;these were entered into the WinBugs (http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/contents.shtml) model code.

estimsure ps havia = 99.

speci as 70

G PCR S 5th ply, the peciew as 9ubpop

was ttly. Thparamthe ex>2000

Bayese relae indiritionalacy mve posner etndence

test action p). Moeparatn wasive cut

estiming ththe inls, rel, the P

depen cutof

indepccuraan andere oe mod

Modensitivirior tes (e.g.,tion torcentaSe froms the ng estwas agxceptical procedures

ostic testsl database containing the results from the indi-M PCR, and SG PCR was sent from the Murtaughto NAHMS staff. The indirect ELISA results weres optical density (OD). The 2 PCR results were

cycle threshold (Ct). For the TM PCR, a differ-etween duplicate tests and the log DNA copies

the reaction based on the standard curve men-uvanendiran et al. (2011) was included. Thelts included the predominant genotype ampli-ssay (A, B, or a mixture) as determined by theint analysis. Sample-level results were linkedod collection records to the farm of origin. Thease was imported into SAS (www.sas.com ver-d merged with selected variables from the bloodection form dataset, the NASS initial swine farmire dataset, and the rst of the two VMO ques-atasets. ELISA test results were analyzed usingt cutoffs (0.3 and 0.4 OD) to distinguish pos-egative samples. Criteria for a TM PCR-positive

a Ct value less than 38 with a value differenceplicates of less than 1 Ct. For the SG PCR, a pos-ted of a Ct value less than 45. The nal datasett the sample level.

ian test populationBayesian latent class analysis, prior estimatespulation (i.e., the U.S. swine herd) exposureand farm characteristics that affect the lev-sure prevalence were researched. To the bestledge, there are no published national esti-

e prevalence of antibodies to PCV2 or of PCV2rm characteristics proposed to affect levels of

PCV2. Herd size was selected as our primaryvariable on which to group positive and neg-sults based on the NAHMS Swine 2006 study,rted herd-size differences in the percentage ofeported PCVAD during the previous 12 months

rst VMO interview. The hypothesis was that would extend to PCV2 exposure levels as well7). Prior to use as a population in Bayesian mod-est results (ELISA, TM PCR and SG PCR) wereated with two categories of herd size: herds

pigs or herds with 2000 pigs in total inven-e 1, 2006. Frequentist methodology (Pearsonsnd Fishers Exact Test) was used on these cross

to determine if the level of exposure as mea-e tests differed between herd sizes and wouldo use in subsequent Bayesian modeling.

ian prior estimates and distribution

timates were expert opinion supplied by theaboratory. The 5th percentile values and theire input into Beta Buster (http://www.epi.

Priorexpoed a90% (wereas lowthe Sand aFinalwas sas lothe smodesligheach with with

2.2.4.Th

of thcondaccurto haGarddepefromtribu(2001run setitiopositlence

Usrun: modedent)PCRsELISAPCRsTest amedition wfor th

2.2.5.Se

ing ppointtribu10 peELISAtion astartieter the eates of the ELISASe, TM PCR Se and Sp and therevalence for herds with 2000 pigs was speci-ng a mode of 95% and a 5th percentile as low as69 and b = 6.19). Prior estimates of the ELISASped as having a mode of 75% and a 5th percentile% (a = 174.47 and b = 58.82). Prior estimates ofe and Sp were specied as having a mode of 99%ercentile as low as 95% (a = 88.28 and b = 1.88).exposure prevalence for herds with >2000 pigsd as having a mode of 99% and a 5th percentile0% (a = 34.17 and b = 1.34). The rationale forulation exposure prevalence only differing inhat the prevalence was expected to differ onlye starting estimate for the Markov chains foreter was the lower bound (e.g., ELISASe = 0.9)ception of the exposure prevalence for herds

pigs which was 0.95.

ian methodtionship between the accuracy measurementsect ELISA and the PCRs were assumed to bely independent. The relationship between theeasurements of the two PCRs was structureditive conditional dependence as described by

al. (2000) and Branscum et al. (2005). This was enumerated as correlations calculatedcuracy covariance estimated from uniform dis-arameters as done previously by Bouma et al.dels containing the indirect ELISA results wereely for the 0.3 and 0.4 OD cutoffs. This rep-

to examine the effect of changing the ELISAoff on the assay accuracy and exposure preva-ates.e herd size population a total of 7 models wasdirect ELISA with each PCR at both cutoffs (4ationship between ELISA and PCRs indepen-CRs together (1 model, relationship betweendent), and all three tests together at bothfs (2 models, relationship between ELISA andendent, relationship between PCRs dependent).cies and exposure prevalences estimates (mean,

95% probability interval) for each subpopula-utput from each model. An example of the codeels can be found in Appendix A.

ling and estimate diagnosticsty testing was done for each model by lower-st accuracy estimates mode by 10 percentage

ELISASe from 0.95 to 0.85), relaxing the beta dis- a 50th percentile and reducing the lower boundge points below the previous lower bound (e.g.,

0.9 to 0.8) and using a uniform beta distribu-prior distribution for exposure prevalence. Theimate for the Markov chains for each param-ain the lower bound (e.g., ELISASe = 0.8) withon of the exposure prevalence. For herds with


>2000 pigs the exposure remained at 0.95 and for herdswith 2000 pigs it remained at 0.9.

A burn in period of 10,001 simulations was used toremove initial samples and the total number of simula-tions (iteraof chains wautocorrelaferent sets oeach estima

3. Results

3.1. Test po

A total oto the labor188 were ofways (e.g., swere testedPCR, and 10and forty sa995 sampleall three tesnot the TMto 998 (Tab

For the Eegorizationa subpopultotal inventlevel (200numbers wquencies infor each mlar, the numhigher ELIS

Based oncame from hto have a hpigs in totatests as a citive) for alsize both thwere signishown). Whcross tabulations, both were signishown).

3.2. Accura

Accuracand SG PCRcentile) seletheir 95% p(Table 2). Thof 95% of thtion in all median estthe correlatPCRs when

Overall, the ELISASe 95% probability interval of 8389%was constant regardless of comparison test(s) combina-tion at the 0.3 cutoff and 7782% at the higher OD cutoff(Table 2).

ese esr the re 3). Th

PCRSthe TM

togethe 2). The medir OD cue highr than 0%) an

higherled we TM

92% ved in86%) t

(Tabstentlyis cases 2 ae estimrobab

a medable 2PI: 38wise. the ELes 2 anom 81

obser(Tableest witling (Ttivity ar all tbetwey as mr PI ofables

all th analy

consiimitedy accu

Popula

e parale-levxposu

in herxceptiand thtivity lencetions) for all models was 100,000. Convergenceas tested by three diagnostic tools: trace plots,tion plots, and comparison of chains with dif-f starting values using Gellman Rubin plots forte (Gelman and Rubin, 1992).

pulation characteristics

f 6234 samples from 185 farms were submittedatory at the University of Minnesota. Of these,

insufcient quantity or were unusable in otherent but not received). Six thousand and forty six

by the indirect ELISA (ELISA), 4147 by the TM00 by the SG PCR. Four thousand one-hundredmples were tested by both ELISA and TM PCR,s by both ELISA and SG PCR, and 993 samples byts. Two samples were tested by the SG PCR but

PCR reducing the number tested by both testsle 1).LISA and TM PCR models with the herd size cat-, 2211 samples came from what was consideredation with higher exposure prevalence (>2000ory) while 1929 were from a lower prevalence0 total inventory) (Table 1). The total sampleere equivalent for each ELISA cutoff, but fre-

the underlying tables supporting the data stepodel in WinBugs were different. In particu-bers of concordant positives were lower at the

A cutoff (Table 1) as would be expected. the Fishers Exact Test, sampled animals thaterds with >2000 pigs were deemed more likely

igher level of exposure than herds with 2000l inventory on June 1, 2006. Additionally, whenount of positive tests (0,1, 2, or all 3 tests pos-l observations were cross tabulated with herde Pearsons chi square and Fishers Exact Testcant at p = 0.01 at both ELISA cut offs (data noten the count of positive and negative tests wereted with the connement and PCVAD popula-the Pearsons chi square and Fishers Exact Testcant at p = 0.01 for both ELISA cut offs (data not

cy estimates

y estimates (Se and Sp) of the ELISA, TM PCR, are presented as the median value (50th per-cted from the posterior distribution along withrobability interval (PI) from that distributione probability interval shows the range of valuese samples selected from the posterior distribu-of the models. Also shown in Table 2 are theimates and associated probability intervals ofions between the accuracy estimates of the two

both are present in the same model.

Thunde(Tablor SGthan were(TablPCRSloweAt thlowe(688of a mode

Th83 toobser(84, aloneconsiin th(TablPCRSthe pwithing (T71% (otherthan (Tablval frvaluePCR any tmodesensi

Fotent fell blowesis (Twhentivitymorehad lfor an

3.3.

Thsamplow elencethe ePCR Sensiprevatimates in Table 2 were relatively unchangedelaxed conditions of the sensitivity modelinge ELISASe median was higher than the TM PCRSe

e when modeled alone with them but higher PCRSe median estimate when all three testser in the original models at the lower OD cutoffe median ELISASe was only higher than the TMan in the sensitivity modeling (Table 3) at thetoff (when only those two tests were modeled).er cutoff the ELISASe median estimate alwaysmedian PCRSe. The ELISASp probability intervald median (74%) remained the same regardless

or lower cut ELISA cutoff or what tests wereith it in Table 2.

PCRSe 95% probability intervals ranged fromand numerically lower median values were

the models that coupled it and the ELISAhan when it was modeled with the SG PCRle 2). The TM PCRSe median estimates were

higher than the SG PCRSe median estimatese in the original and sensitivity modelingnd 3). Sensitivity analysis did not alter the TMates substantially save for a slight expansion ofility interval. The TM PCRSp was relatively highian of 94% (PI: 8898%) in the original model-). Sensitivity analysis reduced this estimate to95%) with the ELISA and 5658% (PI: 3982%)The median TM PCRSp was consistently higherISASp, but not higher than the median SG PCRSpd 3). The SG PCRSe had a 95% probability inter-

to 90% with the numerically highest medianved (87%) when modeled alone with the TM

2). The SG PCRSp was numerically highest ofh a median of 98% (PI: 94100%) in the originalable 2) and its median remained highest in thenalysis Table 3).hree tests the Sp estimates remained consis-en models when using informative priors, butuch as 52 percentage points (in the case of the

the ELISASp, 6816%) in the sensitivity analy-2 and 3). The difference was generally greatestree tests were modeled together in the sensi-sis save for the SG PCRSp which was lower butstent across test combinations. The two PCRs

evidence of a positive conditional dependenceracy measures.

tion prevalence estimates

meter estimates derived by these models wereel exposure prevalence to PCV2 in high andre subpopulations. The median exposure preva-ds with >2000 pigs was 99% in all models withon of the ELISA at its higher cutoff with the TMe SG PCR with TM PCR test pairing (Table 4).analysis generally produced lower exposure

estimates in both herd size subpopulations with


Table 1Two and three-way classication of test results within Herd Size Subpopulationsa for ELISA (OD 0.3 and OD 0.4), TM PCR and SG PCR for the detectionof PCV2 exposure in grow-to-nish pigs in the United States.

ELISA OD 0.3 ELISA OD 0.4

Herd size Herd size

Resultsb >2000 2000 Total >2000 2000 TotalELISA and TM PCR+ + 1625 1360 2985 1495 1250 2745+ 308 287 595 272 257 529 + 224 210 434 354 320 674 54 72 126 90 102 192Total 2211 1929 4140 2211 1929 4140ELISA and SG PCR+ + 337 332 669 305 303 608+ 57 105 162 53 95 148 + 49 76 125 81 105 186 13 26 39 17 36 53Total 456 539 995 456 539 995All three tests+ + + 311 287 598 281 262 543 + + 45 63 108 75 88 163+ + 32 63 95 29 56 85 + 10 15 25 13 22 35+ + 26 44 70 24 41 65 + 4 12 16 6 15 21+ 25 42 67 24 39 63 3 11 14 4 14 18Total 456 537 993 456 537 993

Resultsb >2000 2000 TotalTM PCR and SG PCR+ + 350 358 708+ 78 43 121 + 57 30 87 53 29 82Total 538 460 998

a Farms with a herd size of >2000 are considered more likely to have higher exposure prevalence than those with herd size 2000.b Order of tests: ELISA, TM PCR and SG PCR.

Table 2Bayesian posterior estimates (median (95% probability interval) of pig-level Se and Sp of ELISA, TM PCR and SG PCR for detecting PCV2 exposure ingrow-nish pigs in the United States.*

ELISA With TM PCRa With SG PCRb With botha,b

Cutoff 0.3 Cutoff 0.4 Cutoff 0.3 Cutoff 0.4 Cutoff 0.3 Cutoff 0.4Se 0.88 (0.870.89) 0.81 (0.800.82) 0.86 (0.840.88) 0.79 (0.770.82) 0.85 (0.830.87) 0.79 (0.760.81)Sp 0.74 (0.680.80) 0.74 (0.680.79) 0.74 (0.680.79) 0.74 (0.680.79) 0.74 (0.680.79) 0.74 (0.670.79)

TM PCR With ELISAa With SG PCRb With botha,b

0.3 0.4 N/A 0.3 0.4Se 0.84 (0.830.86) 0.85 (0.840.87) 0.90 (0.870.92) 0.86 (0.840.88) 0.86 (0.840.88)

Se correlation 0.00 (0.010.02) 0.03 (0.020.05) 0.03 (0.020.04)Sp 0.94 (0.880.98) 0.94 (0.880.98) 0.94 (0.880.98) 0.94 (0.870.97) 0.94 (0.870.97)

Sp correlation 0.01 (0.000.03) 0.01 (0.000.03) 0.01 (0.000.03)

SG PCR With ELISAa With TM PCRb With botha,b

0.3 0.4 N/A 0.3 0.4Se 0.84 (0.810.86) 0.84 (0.820.87) 0.87 (0.840.90) 0.83 (0.810.85) 0.83 (0.810.86)

Se correlation 0.00 (0.010.02) 0.03 (0.020.05) 0.03 (0.020.04)Sp 0.98 (0.941.00) 0.98 (0.941.00) 0.98 (0.941.00) 0.98 (0.941.00) 0.98 (0.941.00)

Sp correlation 0.01 (0.000.03) 0.01 (0.000.03) 0.01 (0.000.03)

* All models in this table use the following informative priors as applicable: ELISASe: beta (99.69, 6.19); ELISASp: beta (174.47, 58.82); TM PCRSe: beta(99.69, 6.19); TM PCRSp: beta (99.69, 6.19); SG PCRSe: beta (88.28, 1.88); SG PCRSp: beta (88.28, 1.88); Herds with >2000 Animals Exposure Prevalence: beta(34.17, 1.34); herds with 2000 Animals Exposure Prevalence: beta (99.69, 6.19).

a Based on the assumption of conditional independence between TM PCR or SG PCR and ELISA accuracy estimates (Se and Sp).b Based on the assumption of conditional dependence between TM PCR and SG PCR accuracy estimates (Se and Sp).


Table 3Bayesian posterior estimates (median (95% probability interval) in sensitivity analysis of pig-level Se and Sp of ELISA, TM PCR and SG PCR for detectingPCV2 exposure in grow-nish pigs in the United States.*

ELISA With TM PCRa With SG PCRa With bothb

Cutoff Se 0.77 (0.Sp 0.33 (0.

TM PCR Rb

Se 0.97) Se correla 0.05)

Sp 0.82) Sp correla 130.03

SG PCR CRb

Se 0.97) Se correla 0.05)

Sp 0.91) Sp correla .130.0

* All models le: ELISbeta (7.94, 2.2 eta (11.beta (1, 1); he

a Based on t and ELISb Based on t ccuracy

greater distity interval

4. Discussi

The objenostic perfoPCR tests fosure to PCVtechniquesis no gold and/or expwish to vagold standaa frequentiresearcherswhich reduwith reasonods help eliand obtainunvalidatedin small or lods in this potential offrom smalle

Occasionods and refaccuracy esresults usinparison tesor fecal curesults) or ferent studstudy was f

iates sooral i

ame s the viar it (bility le makas collerties ction a

collecotentiCutoff 0.3 Cutoff 0.4 Cutoff 0.3 0.88 (0.870.90) 0.81 (0.790.84) 0.85 (0.820.88) 0.45 (0.210.84) 0.47 (0.290.84) 0.27 (0.160.48)

With ELISAa With SG PC

0.3 0.4 N/A 0.85 (0.830.91) 0.86 (0.840.95) 0.91 (0.85

tion 0.02 (0.000.71 (0.390.94) 0.71 (0.380.95) 0.58 (0.41

tion 0.06 (0.

With ELISAa With TM P

0.3 0.4 N/A 0.88 (0.820.97) 0.88 (0.820.97) 0.89 (0.83

tion 0.02 (0.000.77 (0.450.96) 0.80 (0.480.96) 0.71 (0.53

tion 0.06 (0 in this table use the following less or non-informative priors as applicab3); TM PCRSp: beta (7.94, 2.23); SG PCRSe: beta (11.58, 2.31); SG PCRSp: brds with 2000 Animals Exposure Prevalence: beta (1, 1).he assumption of conditional independence between TM PCR or SG PCR he assumption of conditional dependence between TM PCR and SG PCR a

ance between medians and more 95% probabil- overlap between the subpopulations (Table 5).

on

ctives of this study were to assess the diag-rmance measures of a new ELISA and two newr PCV2 and determine the prevalence of expo-2 in the U.S. swine population using Bayesian

in the absence of a gold standard. Often therestandard or the one available is cumbersome

alleva humthe shapsto cleThe asampple wpropcollewere(no pensive (Hanson et al., 2003). Researchers that

lidate tests may attempt to create their ownrd and assess diagnostic test accuracies usingst methodology (Opriessnig et al., 2007). These

are also often limited by a small sample sizeces power to determine accuracy estimatesably sized condence intervals. Bayesian meth-minate the need for developing costly standards

accurate estimates of the prevalence from test data in combination with prior knowledgearge sample studies. The use of Bayesian meth-study with a larger sample size allowed for the

more precise estimates than would be obtainedr studies.ally, researchers may apply Bayesian meth-erence pseudogold tests in determining testtimates. This pseudogold may come from testg different biological media than to the com-ts (e.g., virus isolation from nasal secretionslture results to be compared with serologicalbe collected in different time frames using dif-y protocols (Nrette et al., 2008). The currentacilitated by an unusual property of PCV2 that

Virus anadvantage were attemincluding tsurement. Tto extract other wordrather thanbe why samative for PCexposed anbe newly inin the caseadvisable inresponse ti

4.1. Test ac

The postby the dataformed prioexample, w 0.4 Cutoff 0.3 Cutoff 0.4740.81) 0.85 (0.820.87) 0.77 (0.740.80)210.54) 0.25 (0.160.42) 0.30 (0.210.47)

With bothb

0.3 0.40.89 (0.850.96) 0.90 (0.850.96)0.03 (0.000.05) 0.03 (0.000.05)0.56 (0.390.79) 0.56 (0.390.79)

) 0.06 (0.13-0.06) 0.06 (0.130.06)

With bothb

0.3 0.40.88 (0.820.97) 0.88 (0.820.97)0.03 (0.000.05) 0.03 (0.000.05)0.72 (0.530.91) 0.73 (0.540.91)

3) 0.06 (0.000.04) 0.06 (0.130.06)ASe: beta (7.94, 2.23); ELISASp: beta (2.79, 1.96); TM PCRSe:58, 2.31); herds with >2000 Animals Exposure Prevalence:

A accuracy estimates (Se and Sp). estimates (Se and Sp).

me of these considerations. The virus generatesmmune response without viral clearance so thaterum sample contains virus particles (or per-rus itself) in tandem with antibodies intendedMurtaugh, pers. comm.; Vincent et al., 2003).to obtain virus and antibodies from the samees us sure that the animal from which the sam-ected was exposed to PCV2. Besides the uniqueof the immunologic response to PCV2 the datalso beneted from timing because the samplested before the rst PCV2 vaccines were releasedal cross-reactivity).

d antibodies from the same sample is anbut may also be a disadvantage since wepting to estimate exposure prevalence and areests more suited to infection prevalence mea-his could introduce bias by way of attemptingestimates from mixed latent populations. Ins, a population of infected pigs based on PCR

exposed based on ELISA results. This bias mayples were occasionally positive for ELISA, neg-R or vice versa. Occasionally animals may bed clear the virus in the case of the former orfected (reinfected) prior to humoral response

of the latter. Serial blood testing would be the case of discordant results if immunologic

ming is suspected.

curacies

erior estimate for Se of all three tests was driven regardless of whether informative or unin-rs (in the sensitivity analysis) were used. Forhether both the ELISASe and TM PCRSe started

C. H

aley et

al. /

Preventive V

eterinary M

edicine 101 (2011) 79 88

85

Table 4Bayesian posterior estimates (median (95% probability interval) of pig-level PCV2 Exposure Prevalence in grow-nish pigs in the United States.*

ELISA with TM PCRa ELISA with SG PCRa SG PCR with TM PCRb All threea,b

Cutoff 0.3 Cutoff 0.4 Cutoff 0.3 Cutoff 0.4 N/A Cutoff 0.3 Cutoff 0.4Higher exposed prevalence 0.99 (0.981.00) 0.96 (0.940.98) 0.99 (0.971.00) 0.99 (0.961.00) 0.95 (0.920.98) 0.99 (0.981.00) 0.99 (0.981.00)Lower exposed prevalence 0.97 (0.950.98) 0.98 (0.960.99) 0.95 (0.920.97) 0.94 (0.910.97) 0.91 (0.880.94) 0.97 (0.940.98) 0.96 (0.930.98)

* All models in this table use the following informative priors as applicable: ELISASe: beta (99.69, 6.19); ELISASp: beta (174.47, 58.82); TM PCRSe: beta (99.69, 6.19); TM PCRSp: beta (99.69, 6.19); SG PCRSe: beta(88.28, 1.88); SG PCRSp: beta (88.28, 1.88); herds with >2000 Animals Exposure Prevalence: beta (34.17, 1.34); herds with 2000 Animals Exposure Prevalence: beta (99.69, 6.19).


Table 5Bayesian posterior estimates (median (95% probability interval) in sensitivity analysis of pig-level PCV2 Exposure Prevalence in grow-nish pigs in the United States.*

ELISA with TM PCRa ELISA with SG PCRa SG PCR with TM PCRb All threea,b

Cutoff 0.3 Cutoff 0.4 Cutoff 0.3 Cutoff 0.4 N/A Cutoff 0.3 Cutoff 0.4Higher exposed prevalence 0.98 (0.851.00) 0.91 (0.730.97) 0.94 (0.801.00) 0.95 (0.811.00) 0.91 (0.800.99) 0.94 (0.821.00) 0.93 (0.821.00)Lower exposed prevalence 0.93 (0.780.98) 0.95 (0.790.99) 0.81 (0.580.92) 0.81 (0.610.92) 0.77 (0.650.89) 0.79 (0.660.91) 0.79 (0.660.90)

* All models in this table use the following less or non-informative priors as applicable: ELISASe: beta (7.94, 2.23); ELISASp: beta (2.79, 1.96); TM PCRSe: beta (7.94, 2.23); TM PCRSp: beta (7.94, 2.23); SG PCRSe:beta (11.58, 2.31); SG PCRSp: beta (11.58, 2.31); herds with >2000 Animals Exposure Prevalence: beta (1, 1); herds with 2000 Animals Exposure Prevalence: beta (1, 1).



with modes of 85% (with a greater probability of a valueabove or below that) or 95% the median posterior esti-mates did not change considerably and were lower thanprior belief when modeled together at the 0.3 OD ELISAcutoff. A shoff reducedor SG PCRSthere is a swreduction ctive and neposterior esthan the othsidering higto tell whictic sensitiviexposure (vacross modat its lowercutoff, the Enations savThe medianest of all metogether in ever, its vaestimate st

Converstests was imedian estthe prior dprior informby as muchother moreable analytwhich wouthere was nBiosystemsspecicity ivalues and best havingthen (7180ysis at eithe

Increasilogic tests model, appfor the ELImodeled wof negativesamples/anels enough resulted inthe ELISASpreduced in sensitivity aspecic notPCV2 basedproducts.

Our prioData drivenbelief for alprior distribmate possib

the potential of a nearly absent latent negative populationfor PCV2 exposure created wide probability intervals inposterior distribution sampling. This indicates the need fortest application in truly positive and negative PCV2 animals

ture st

e ELIendena (Geobetweor mowhen nced

sure stost corformfore, i

e betwndench calcuwo acsed (B

l covarndicat).

Selectiocatio

hethes in Uertz,

is likelre sub

(the r) bec

popu pathoe of mh (Hur

addre(accuropulatlenceixed m

and itive Txposurded asm int

size w of signe that t on th

size sh a pose herate teguishlenceas raisift to a more conservative 0.4 OD ELISA cut- the ELISASe but did not affect the TM PCRSee. There is commonly a reduction in Se whenitch to a higher cutoff in tests of this type. This

an occur when the latent distributions of posi-gative results overlap (Dohoo et al., 2003). Thetimates of the SG PCRSe were more data drivener two tests when using informative priors con-h starting parameter estimates. It is difculth of the three tests has the greatest diagnos-ty. Given that the ELISA is designed to measureia antibodies) and considering its performanceels, its Se is likely the best of all three tests

cutoff. In the original modeling, at the lowerLISASe median was highest across test combi-e when all three tests were modeled together.

ELISASe was lower than the SG PCRSe and low-dian values when the three tests were modeledthe sensitivity analysis at its lower cutoff. How-lue PI had the smallest range indicating moreability compared to other tests.ely, the posterior estimate for Sp of all threenuenced by prior information. The posteriorimates of Sp were often very near the mode ofistribution. In the case of the ELISASp, relaxingation reduced the median posterior estimates

as 49 percentage points when modeled with analytically specic PCR tests. The most reli-ic Sp may be obtained from the real-time PCRsld simply cycle without attaining a threshold ifothing for the primers to adhere to (Applied

, 2008). The two PCRs have greater diagnosticn all cases over the ELISA considering medianPI estimates. Of the two, the SG PCRSp is likely

higher median values and a smaller range of%) in the original modeling and sensitivity anal-r ELISA OD cutoff.

ng the cutoff usually increases the Sp of sero-although this was not the case here. In eachroximately 1417% of samples were negativeSA test in combination with any test it wasith. It is also possible that the small number

results and the number of latent unexposedimals in the data did not provide these mod-power to adequately estimate the ELISASp. This

the large drop in the sensitivity analysis forposterior estimates. The two PCRSp were alsothis analysis. The SG PCRSp fell by the least innalysis. The SG PCRSp may be bolstered by being

only for PCV2 but for genotypes A and B of on the melting temperature prole of amplied

r beliefs as to specicity were too optimistic. posterior estimates were lower than priorl tests specicities when the latent estimatesutions were wide enough to include lower esti-ilities (sensitivity analysis). This coupled with

in fuwell.

Thindepmedicies two (and inueexpothe mtic peTheredencdepewhicthe twas usmalnot i1988

4.2. classi

WuitouManksure or molenceearliemarysomeposithealt

Toysis subpprevaear m(0.3A posfor einclurandoHerdleveldenceffecHerdgiven

Thestimdistinprevaoff wudy as we believe the tests themselves work

SA and any PCR should be conditionallyt since they measured different biologicalrgiadis et al., 2003). Conditional dependen-

en test accuracy estimates often arise whenre) tests evaluate similar substrates (e.g., DNA)the probability of the results on one test isby those of another, depending on disease oratus. Positive conditional dependencies are alsommon type of relationship between diagnos-

ance measures in biology (Gardner et al., 2000).t was assumed that positive conditional depen-een the two PCRs existed, but the degree of thate was difcult to estimate a priori. A methodlates parameters of covariance distributions ofcuracy estimates from a uniform distributionouma et al., 2001; Branscum et al., 2005). Theiance/correlation estimates in Tables 4 and 5 doe a departure from test independence (Cohen,

n of population for exposure leveln

r the common assumption that PCV2 is ubiq-.S. grower/nisher herds (Finsterbusch and

2009) is true or not, the prevalence for expo-y high. Therefore nding a population with twopopulations with different levels of true preva-rst assumption of latent class analysis discussedomes difcult. Herd size was used as the pri-lation. Herd size is a nonspecic risk factor forgens and is likely a composite or part of a com-any factors reecting farm management andnik et al., 1994).ss the second assumption of latent class anal-acy of both tests should be constant acrossions regardless of the disease or exposure

in the subpopulations), a series of general lin-odels (GLMM) were run at each ELISA cutoff0.4) in a previous study using the same data.M PCR was the event of interest as a surrogatee status. Herd size as well as ELISA result was

xed effects and the farm was modeled as aercept due to blood samples clustered within it.as not associated with the outcome at the 5%icance. We take these results to provide evi-

our main population classication likely has noe relationship of results between the two tests.ould not affect the odds of seeing a positive PCRitive ELISA (Haley et al., 2009).d size population appeared to be adequate tost accuracies and perhaps more importantly to

between higher and lower levels of exposure in all models but one. When the ELISA cut-ed to 0.4 and modeled with the TM PCR, the


subpopulation assumed to have a lower exposure preva-lence (herd size 2000) had a higher median value thanof the subpopulation assumed to have a higher exposureprevalence. This is what would be expected with somedegree of ohigher and sensitivity ties; our poa large parple sizes. Wuninformataround theof prevalenmodels incinclusion ofcic accuraestimates msure preval

5. Conclus

Our claacross modcient for manimals. Th100% in nnot be 100%had the grean estimateas being spit has the bthe ELISA sobe used priused to assetwo PCRs, tsurements.new PCRs winterpretatigreater accexposure phigh. The SGnegative ashigh vaccinutility of thconnectionhave a protcine strainsfuture.

Acknowled

The autto Drs. Ian into the anUSDAAPHas well as cples and mto enter the

Appendix A. Sample code

WinBugs Code for accuracy estimates of all three testsand true exposure prevalence using the ELISA cutoff 0.3

he Her

el;{K, 1:K,K, 1:K,,1,1] *(1-Sp,2,1] i1)*(1-,1,2] 0.95SpSGpcr dbeta(88.28, 1.88) ## Mode=0.99, 95% sureSpSGpcr > 0.95Z dbern(tau1)pi1star dbeta(34.17, 1.34) ## Mode=0.99, 95% sure pi2> 0.90, Higher Exposedpi1

Diagnostic performance measures of ELISA and quantitative PCR tests for porcine circovirus type 2...

Documents

Transcript of Diagnostic performance measures of ELISA and quantitative PCR tests for porcine circovirus type 2...