Comparability of different ELISA’s on the detection of ... · Results serum panel (3)...
Transcript of Comparability of different ELISA’s on the detection of ... · Results serum panel (3)...
![Page 1: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/1.jpg)
CRL-Salmonella
Petra Berk | 26 February 2009
Comparability of different ELISA’s on the detection of Salmonella spp. antibodiesin meat juice and serum of pigs
![Page 2: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/2.jpg)
Background• Baseline survey on the prevalence of Salmonella spp. in
slaughter pigs (2006/668/EC)• New – the possibility to use serological methods on meat
juice • 10 NRLs-Salmonella using their own method
Two studies organised by the CRL-Salmonella1. Duplicate analysis study2. Interlaboratory comparison study at the end of the baselinestudy
![Page 3: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/3.jpg)
Information on ELISAs used
Labcode ELISA Data- ± +
1 Salmotype PigScreen (LDL) OD % <10 >10 and <20 >202 Salmotype PigScreen (LDL) OD % <10 >10 and <20 >203 HerdCheck Swine Salmonella (IDEXX) OD % <10 >10 and <20 >204 HerdCheck Swine Salmonella (IDEXX) OD % <10 >105 VetSign Porcine Salmonella (Guildhay) S/P ratio <0.10 >0.10 and <0.25 >0.256 Salmotype PigScreen (LDL) OD % <10 >10 and <20 >207 In-house OD % <40 >408 HerdCheck Swine Salmonella (IDEXX) OD % <15 >159 In-house OD % <20 >20
Cut-off values used
![Page 4: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/4.jpg)
Calculation of OD % per ELISA• S/P ratio
• HerdCheck Swine Salmonella
• Salmotype PigScreen
• In-house ELISA’s- Reference sera, regression model
%100*5.2
/% PSOD =
NCPC
NCSample
ODODODOD
PS−
−=/
1.72*/% PSOD =
![Page 5: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/5.jpg)
CRL-Salmonella
Duplicate analyses of meat juicesamples collected during the baseline study on fattening pigs
![Page 6: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/6.jpg)
Materials and Methods• Selection of 60 meat juice samples with different OD%
• 120 µl meat juice sample send to the CRL-Salmonella
• Samples were collected by the CRL and stored at -20°C
• Samples were send to the Animal Health Service (GD) in the Netherlands
• GD analysed all samples with HerdCheck Swine Salmonella
• Comparison results GD with NRLs
![Page 7: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/7.jpg)
57606060606060606060total
1020101031311102110>50
103981551051040-50
771091641031030-40
10910109121310111020-30
101011102213181091010-20
1011101324119101110<10
1098764321
Laboratory codesselection criteria OD %
S/P ratio 5<0.1 14
0.1-0.2 120.2-0.3 60.3-0.4 40.4-0.5 1
>0.5 23total 60
![Page 8: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/8.jpg)
Results
-20
0
20
40
60
80
100
120
140
160
1 7 47 2 34 27 38 50 42 31 33 8 45 57 29 36 41 54 14 52 49 59 48 21 39 19 28 55 24 9 17 22 43 46 51 6 56 44 20 58 15 60 30 13 32 18 12 26 25 53 4 37 5 23 3 11 40 35 16 10
sample
OD
%
NRLCRL
Labcode 1
-20
0
20
40
60
80
100
120
140
25 4 23 51 43 47 16 54 34 1 22 46 12 15 30 2 41 44 53 42 37 33 5 56 59 28 29 24 9 50 10 52 3 14 36 20 31 57 32 17 45 11 48 39 26 58 55 21 6 18 7 40 27 60 38 13 49 35 19 8
sample
OD
%
NRLCRL
Labcode 2
-1
0
1
2
3
4
5
6
7
8
9
1 2 3 8 56 4 7 5 6 60 58 59 57 55 49 54 10 12 11 15 51 48 52 53 14 16 47 50 13 41 17 19 20 18 42 22 40 45 46 43 44 24 21 23 25 27 26 28 34 38 32 29 37 36 31 30 39 33 35
sample
SP ra
tio
NRLCRL
Labcode 5
![Page 9: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/9.jpg)
Dependent t-testAverage OD%Labcode ELISA
NRL CRL*
1 Salmotype PigScreen 42.84 37.43 0.047
2 Salmotype PigScreen 31.41 18.13 1.31*10-7
3 HerdCheck Swine 30.12 29.15 0.74
4 HerdCheck Swine 31.70 37.60 0.039
5 VetSign Porcine* 0.692 1.433 1.4*10-7
6 Salmotype PigScreen 14.87 6.87 0.0001
7 In-house 31.06 35.47 0.11
8 HerdCheck Swine 31.44 39.37 0.029
9 In-house 38.09 31.71 0.018
10 Salmotype PigScreen 36.08 19.46 0.0004
P value
* CRL used HerdCheck Swine ELISA
![Page 10: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/10.jpg)
CRL-Salmonella
Interlaboratory comparisonstudy on serological methods
![Page 11: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/11.jpg)
Materials and Methods• 40 sera were sent to the NRLs for serological analysis
- 2x Salmonella negative pigs- 2x pigs inoculated with Yersinia enterocolitica O3-/O9-- 36 pigs inoculated with Salmonella spp.
• 4x S. Brandenburg (group B)• 24x S. Typhimurium (group B)• 2x S. Livingstone (group C1)• 4x S. Goldcoast (group C2)• 2x S. Panama (group D)
• All NRLs interpreted their results using a cut-off value used routinely
![Page 12: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/12.jpg)
Results specificity panelNo. Description exp 1 2 3 4 5 6 7 8 9 10S-3 negative - - - - - - - - - - ±S-8 negative - - - - - - - - - - ±S-2 Y. enterocolitica O3-O9- - - - - - - - - - - +S-9 Y. enterocolitica O3-O9- - - - - - - - - - - ±S-1 S . Goldcoast - - - - - - - - - - -S-7 S . Goldcoast - - - + - - - - - - ±
S-21 S . Goldcoast - - - - - - - - - - ±S-32 S . Goldcoast - - - - - - - - - - ±
participant
No. Description 1 2 3 4 5* 6 7 8 9 10S-3 negative 1 0 2 8 0.018 0 -1 9 -6 12S-8 negative 1 1 3 4 0.018 0 4 7 -5 12S-2 Y. enterocolitica O3-O9- 2 3 7 7 0.044 1 2 6 -4 25S-9 Y. enterocolitica O3-O9- 1 0 2 5 0.032 0 -4 5 -8 12S-1 S . Goldcoast 0 0 3 5 0.009 0 -3 12 -8 10S-7 S . Goldcoast 0 0 22 9 0.055 0 -3 9 -6 10S-21 S . Goldcoast 0 0 6 6 0.030 0 -5 8 -8 11S-32 S . Goldcoast 0 0 1 0 0.009 0 -6 -1 -9 13
participant
![Page 13: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/13.jpg)
Results serum panel (1)
No. Description exp 1 2 3 4 5 6 7 8 9 10S-6 S . Brandenburg + - - ± + - - - ± - ±
S-26 S . Brandenburg + + ± + + - + - ± - +S-33 S . Brandenburg + + + + + + + + + + +S-34 S . Brandenburg + + + + + + + + + + +S-4 S . Livingstone + - - - + - - - ± - +
S-10 S . Livingstone + + + + + + + - ± + +S-20 S . Panama + - - - - - - - - - +S-37 S . Panama + + + + + ± + - + + +
participant
No. Description 1 2 3 4 5* 6 7 8 9 10S-6 S . Brandenburg 0 1 11 18 0.044 0 0 20 -5 17
S-26 S . Brandenburg 21 16 21 27 0.100 22 1 15 9 71S-33 S . Brandenburg 73 72 110 117 0.588 67 63 88 88 118S-34 S . Brandenburg 68 51 79 86 0.475 59 43 62 76 100S-4 S . Livingstone 4 4 7 23 0.077 3 3 19 -2 37
S-10 S . Livingstone 54 47 29 56 0.411 52 12 36 28 108S-20 S . Panama 1 0 4 6 0.032 0 -4 4 -8 21S-37 S . Panama 53 43 101 103 0.209 45 35 84 54 99
participant
![Page 14: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/14.jpg)
Results serum panel (2)
No. Description exp 1 2 3 4 5 6 7 8 9 10S-5 S . Typhimurium + + + + + + + + + + +
S-11 S . Typhimurium + - - - - - - - - - +S-12 S . Typhimurium + + + + + + + - + + +S-13 S . Typhimurium + + + + + + + + + + +S-14 S . Typhimurium + + ± ± + ± + - - - +S-15 S . Typhimurium + + + + + + + + + + +S-16 S . Typhimurium + + + + + + + + + + +S-17 S . Typhimurium + - - - + - ± - - - +S-18 S . Typhimurium + + + + + + + + + + +S-19 S . Typhimurium + + + + + + + + + + +S-22 S . Typhimurium + + + + + + + - + - +S-23 S . Typhimurium + + + + + ± ± - + - +S-24 S . Typhimurium + + + + + ± + - ± - +S-25 S . Typhimurium + + + + + + + - + + +S-27 S . Typhimurium + + ± + + ± + - ± - +S-28 S . Typhimurium + + + + + + + - + + +S-29 S . Typhimurium + + + + + + + - + + +S-30 S . Typhimurium + - - ± + - - - - - +S-31 S . Typhimurium + + + + + + + - + + +S-35 S . Typhimurium + + + + + + + - + + +S-36 S . Typhimurium + ± ± ± + - ± - - - +S-38 S . Typhimurium + ± ± + + ± ± - ± - +S-39 S . Typhimurium + - - - - - - - - - +S-40 S . Typhimurium + + ± + + ± + - ± + +
participant
![Page 15: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/15.jpg)
Results serum panel (3)
No. Description 1 2 3 4 5* 6 7 8 9 10S-5 S . Typhimurium 78 97 109 126 1.349 81 93 81 77 133
S-11 S . Typhimurium 4 3 3 3 0.071 3 -1 6 -3 45S-12 S . Typhimurium 39 42 56 82 0.303 36 34 43 34 118S-13 S . Typhimurium 70 83 107 121 1.101 74 83 79 76 119S-14 S . Typhimurium 22 18 16 24 0.175 22 19 14 13 75S-15 S . Typhimurium 49 38 62 87 0.357 46 47 59 40 105S-16 S . Typhimurium 88 89 109 125 1.178 72 95 78 74 136S-17 S . Typhimurium 9 8 8 19 0.060 10 4 12 5 59S-18 S . Typhimurium 52 42 73 90 0.335 51 53 47 47 104S-19 S . Typhimurium 59 53 56 77 0.507 52 57 45 35 104S-22 S . Typhimurium 39 33 72 87 0.360 34 24 53 14 88S-23 S . Typhimurium 21 20 65 81 0.217 20 15 53 10 109S-24 S . Typhimurium 25 22 35 47 0.194 23 20 33 12 87S-25 S . Typhimurium 48 37 84 97 0.292 47 37 64 36 102S-27 S . Typhimurium 23 20 34 39 0.169 23 17 32 15 78S-28 S . Typhimurium 42 41 53 61 0.302 33 23 49 43 115S-29 S . Typhimurium 48 48 46 62 0.316 46 33 43 60 99S-30 S . Typhimurium 3 3 13 14 0.065 3 0 11 -3 27S-31 S . Typhimurium 40 43 49 62 0.266 40 31 44 32 107S-35 S . Typhimurium 48 35 35 46 0.254 40 30 26 35 97S-36 S . Typhimurium 13 11 14 15 0.060 13 10 13 6 65S-38 S . Typhimurium 15 13 28 30 0.116 12 13 25 11 67S-39 S . Typhimurium 1 0 7 7 0.040 0 -2 7 -4 25S-40 S . Typhimurium 30 20 31 42 0.147 26 16 34 20 79
participant
![Page 16: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/16.jpg)
80%63%63%40%75%60%93%75%68%78%accuracy
40404040404040404040total
32252516302437302731number correct
97%53%53%25%69%50%91%72%59%72%Sensitivity
32323232323232323232total positives
3117178221629231923number positives
13%100%100%100%100%100%100%88%100%100%Specificity
8888888888total negatives
1888888788number negatives
10987654321
Labcode
Analysis qualitative results
![Page 17: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/17.jpg)
Analysis quantitative Results• Receiver Operating Characteristics (ROC) plots
- Sensitivity plotted against the specificity at cut-off’s for the whole range of the test
- Area below the curve is proportional to the accuracy of the test • 0.5 = random• 1 = perfect
Labco de 3
0
0,2
0,4
0,6
0,8
1
1,2
0 0,2 0,4 0,6 0,8 1 1,2
sens
spec
High cut-off
Low cut-off
![Page 18: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/18.jpg)
labcode ROC-area S.E.M.
1 0.96 0.028
2 0.96 0.030
3 0.93 0.044
4 0.95 0.034
5 0.98 0.020
6 0.95 0.035
7 0.93 0.042
8 0.93 0.042
9 0.97 0.025
10 0.99 0.014
All NRLs performedvery good and alltests are able todetect the truestatus of the sample, however at different cut-off values
![Page 19: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/19.jpg)
Results per ELISA method (1)• HerdCheck Swine Salmonella ELISA from IDEXX
-20
0
20
40
60
80
100
120
140S-
1S-
2S-
3S-
4S-
5S-
6S-
7S-
8S-
9S-
10S-
11S-
12S-
13S-
14S-
15S-
16S-
17S-
18S-
19S-
20S-
21S-
22S-
23S-
24S-
25S-
26S-
27S-
28S-
29S-
30S-
31S-
32S-
33S-
34S-
35S-
36S-
37S-
38S-
39S-
40
OD%
labcode 3 labcode 4 labcode 8 cut-off
Sensitivity: 89%
Specificity: 92%
Accuracy: 89%
Sensitivity: 74%
Specificity: 96%
Accuracy: 78%
Cut-off = 10 Cut-off = 20
![Page 20: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/20.jpg)
Results per ELISA method (2)• Salmotype PigScreen from Labor Diagnostik Leipzig
-20
0
20
40
60
80
100
120
140
S-1
S-2
S-3
S-4
S-5
S-6
S-7
S-8
S-9
S-10
S-11
S-12
S-13
S-14
S-15
S-16
S-17
S-18
S-19
S-20
S-21
S-22
S-23
S-24
S-25
S-26
S-27
S-28
S-29
S-30
S-31
S-32
S-33
S-34
S-35
S-36
S-37
S-38
S-39
S-40
OD
%
lacode 1 labcode 2 labcode 6 labcode10 cut-o ff
Sensitivity: 84%
Specificity: 75%
Accuracy: 83%
Sensitivity: 77%
Specificity: 97%
Accuracy: 81%
Cut-off = 10 Cut-off = 20
![Page 21: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/21.jpg)
Conclusions• General
- Different ELISA’s were used by different NRLs- Different cut-off values are used
• Duplicate analysis study- In the duplicate analysis study the serological results of the selected
meat juice samples of 9 out of 10 NRLs is different from that of the CRL
- Meatjuice is not homogeneous material- Differences in matrix because of extra thaw-freeze step- Inexperience of some laboratories with meatjuice
![Page 22: Comparability of different ELISA’s on the detection of ... · Results serum panel (3) No.Description 1234 5* 6789 10 S-5 S.Typhimurium 78 97 109 126 1.349 81 93 81 77 133 S-11 S5](https://reader034.fdocuments.us/reader034/viewer/2022042211/5eb2972a6192945bcb5787bb/html5/thumbnails/22.jpg)
Conclusions• Interlaboratory comparison study
- Quantitative ROC analysis indicated that all NRLs performed verygood and all tests are able to detect the true status of the sample, however at different cut-off values
- The NRL with labcode 7 had found the most negative samples, this is due to the high cut-off value used by this NRL (OD % > 40)
- The OD% of the NRL with labcode 10 is in general higher than that of other NRLs in both studies
- ELISA methods were comparable between different NRLs usingserum samples