ISTA’s Position, Experience and Activities in the Field of Biotech Trait Detection ISTA GMO TF...
-
Upload
winifred-dawson -
Category
Documents
-
view
217 -
download
3
Transcript of ISTA’s Position, Experience and Activities in the Field of Biotech Trait Detection ISTA GMO TF...
ISTA’s Position, Experience and Activities in the Field of Biotech Trait Detection
ISTA GMO TF
B.R. Kahlert
S. Grégoire, C. Haldemann, K. Remund, J.L. Laffont
and many others
International Seed Testing Association
Outline
What is ISTA?
ISTA and of GM Seed Testing
Accreditation
Proficiency Tests
Performance Data Evaluation
Audit
ISTA can help, ISTA helps to share
Workshops, Seedcalc, Website
Objectives of the International Seed Testing Association
• The primary purpose of the Association is to develop, adopt and publish standard procedures for sampling and testing seeds, and to promote uniform application of these procedures for evaluation of seeds moving in international trade.
• The secondary purposes of the Association are actively to promote research in all areas of seed science and technology, to participate in conferences and training courses aimed at furthering these objectives, and to establish and maintain liaison with other organisations having common or related interests in seed.
More than:• 160 laboratories,• 200 personal members, • originating from 70 countries supporting the
association (Government mandates)
ISTA: Laboratories Worldwide
ISTA’s major Products & Services
• International Rules for Testing Seed– Sampling, Purity, Other Seeds, Germination,
Tetrazolium, Seed Health, Species and Variety Testing, Moisture, Coated Seed, Vigour, X-ray
– seeds of agricultural & vegetable; flowers, spice, herb & medicinal; tree & shrubs
• Proficiency Test Programme • Laboratory Accreditation Programme• International Seed Lot Certificates
ISTA’s Vision & Strategy inTesting of GM Seeds
• ISTA Seed Testing Laboratories are competence centres for detection, identification and quantification of GM seeds
• ISTA focuses its activity on developing and maintaining a system that ensures reliability and accuracy of results all over the world, when testing for GM seeds – Performance Based Approach
defined in the Position Paper:
ISTA’s Strategy Regarding Methods for the Detection, Identification and Quantification of Genetically Modified Seeds in Conventional Seed Lots
adopted by the Executive Committee Nov. 2001
ISTA’s Actions in Testing of Seed with Specified Trait(s)
1. Chapter in ISTA Rules for testing of seeds with specified trait(s)
2. Implementation of Laboratory Accreditation under Performance Based Approach
3. Organising proficiency tests on detection and quantification of GM seeds in conventional seed
4. Establishing a system for Performance Data Evaluation for the Presence of Seed with Specified Trait(s) in Seed Lots and Specified Trait Purity
5. Provide technical help for the laboratories (training courses, workshops, Seedcalc7, ...)
1. Chapter in the ISTA Rules
• Rules for the testing of seeds with specified traits including the detection, identification and quantification of GM seeds came into force in February 1st, 2006
• The Rules do not contain specific methods, but define the general principles in testing and reporting results of such tests
• Laboratories having methods for GM seed testing in their scope of accreditation can report the test results on the International Seed Analysis Certificate
2. Conditions for ISTA Accreditation under the Performance Based Approach
Laboratory ...• Has established method(s)• Defines scope of accreditation• Participates successfully in the ISTA Proficiency
Test Programme• Submits performance data• Has an audit
3. Organisation of Proficiency Tests
AIMS• to evaluate the performance of participating
laboratories• to collect data to be used to set performance
standards• To increase experience for ISTA and participating
laboratories
Satisfactory performance in proficiency tests is a prerequisite for attaining or maintaining the status as an accredited laboratory
Laboratories can participate to ISTA proficiency tests, even if they do not seek accreditation
Design of the 6th Proficiency Test
• crop: Brassica napus L.• # of samples per lab: 10• # of negative samples: 3• # of positive samples: 7
0.3%: 20.6%: 31.2%: 2
• event: GT73• sample size: 3300 seeds• Qualitative and Quantitative tests• the laboratory chooses the method
Qualitative Test: The detection of GM seeds in samples
49 labs classified each of the 10 samples as either positive or negative.
4 laboratories reported false results. These are 8% of the laboratories.
Sample classified as false...
98
147
97
342
147
489
1
1
2
4
8
12
0% 20% 40% 60% 80% 100%
1.2% GM seeds
0.6% GM seeds
0.3% GM seeds
Positive samples
Negative samples
All samples
% of samples tested
# of samples reported as false
2.4%5.4%1.2%2.0%0.7%1.0%
Sub-sampling Quantification
Laboratory’s estimated results and its mean
Es
tim
ate
of
% G
M
se
ed
s
12 laboratories reported the test plan...
• the size of sub-samples (in number of seeds)
• the number of sub-samples tested
• the number of positive sub-samples
used to compute the result
Labs# of sub-samples
size ofsub-sample
1 ~300 102 ~90 30
3, 4 ~30 ~1105, 6 20-25 ~1507, 8 15 210-220
9, 10, 11 5-10 330-55012 2-3
Spiking level 0.3% Spiking level 0.6% Spiking level 1.2%
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3.0
1 2 3 4 5 6 7 8 9 10 11 12
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3.0
1 2 3 4 5 6 7 8 9 10 11 12
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3.0
1 2 3 4 5 6 7 8 9 10 11 12
Quantitative Test
23 laboratories reported the GM content of the positive test samples in % of GM seed by number or by mass or in % DNA copies
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
% G
M s
ee
ds
Spiking level (%) Mean (%)
Standarddeviation
Variationcoeff.
0.3/0.33 0.38 0.22 59%0.6/0.67 0.80 0.40 50%1.2/1.35 1.57 0.90 58%
Summary PT06
PT06 - Qualitative Results:• The percentage of laboratories reporting correct
results for all samples is higher as in the previous tests
• >90% of labs made no missclassification, • >97% of samples are reported correctly
PT06 - Quantitative Results:• The results show similar variation for the the spiking
level (VC = ~50%)• Most laboratories overestimated the true level
Design of the ISTA Proficiency TestsPT01 PT02 PT03 PT04 PT05 PT06
# of samples per lab
30 10 12 12 12 10
negative 12 3 3 3 3 3 positive 18 7 9 9 9 7
0.1% - - - 3 - -0.2% - - 3 - 2 -0.3% 20.5% - - - 3 3 -0.6% 30.7% - 3 - - - -1.0% 18 - - 3 3 -1.2% 21.4% - 4 - - - -1.5% - - - - 3 -2.0% - - 3 - - -4.0% - - 3 - - -
Events T25/Mon810 (maize)
Mon810 (maize)
T25/Mon810 (maize)
GTS-40-3-2 (soybean)
GTS40/A2704 (soybean)
GT73(rapeseed)
Sample size
300 3000 1500 3000 3000 3000
Qualitative X X X X X XSub-sampl. - (X) (X) X X XQuant. test - (X) (X) X X X
110 laboratories from 42 countries
PT01 – PT06: Qualitative Tests
Percentage of laboratories reporting results correctly or falsely:
Percentage of samples reported correctly or falsely:
PT01: maize, MON810 & T25; 1%
PT02: maize; MON810; 0.7% & 1.4%
PT03: maize; MON810 & T25; 0.2%, 2% & 4%
PT04: soybean; GTS40; 0.1%, 0.5% & 1.0%
PT05: soybean; GTS40 & A2704;
0.2%, 0.5%, 1.0%, 1.5%
PT06: rapeseed; GT70; 0.3%, 0.6% & 1.2%
01020
3040506070
8090
100
PT01 PT02 PT03 PT04 PT05 PT06
% la
bora
torie
s
correct results false results
false negative false positive
both false positive and false negative
0102030405060708090
100
PT01 PT02 PT03 PT04 PT05 PT06
% la
bora
torie
s
PT01 – PT06: Quantitative Test
(Overall) mean of the quantitative test results for each spiking level
0.00
1.00
2.00
3.00
4.00
5.00
6.00
0.1 0.2 0.2 0.3 0.5 0.5 0.6 0.7 1.0 1.0 1.0 1.2 1.4 1.5 2.0 4.0
GTS40 MON810+T25
GTS40 GT70 GTS40 GTS40 GT70 MON810+T25
GTS40 GTS40 A2704 GT70 MON810+T25
GTS40 MON810+T25
MON810+T25
soya maize soya rape soya soya rape maize soya soya soya rape maize soya maize maize
PT04 PT03 PT05 PT06 PT04 PT05 PT06 PT02 PT04 PT05 PT05 PT06 PT02 PT05 PT03 PT03
% m
ean
GM
con
tent
PT01 – PT06: Quantitative Test
Variation coefficient of the quantitative test results
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.1 0.2 0.2 0.3 0.5 0.5 0.6 0.7 1.0 1.0 1.0 1.2 1.4 1.5 2.0 4.0
GTS40 MON810+T25
GTS40 GT70 GTS40 GTS40 GT70 MON810+T25
GTS40 GTS40 A2704 GT70 MON810+T25
GTS40 MON810+T25
MON810+T25
soya maize soya rape soya soya rape maize soya soya soya rape maize soya maize maize
PT04 PT03 PT05 PT06 PT04 PT05 PT06 PT02 PT04 PT05 PT05 PT06 PT02 PT05 PT03 PT03
% m
ean
GM
con
tent
Summary PT01-PT06
Qualitative Results:Better results when• only one transgenic event involved• transgenic events are well known
Better results for negative samples than for positive samples
Quantitative Results:Reported values close to the true value• when spiking level between 0.2% and 1.5%• when transgenic event and reference material are well known
How to evaluate the performance of laboratories in ISTA Proficiency Tests?
A No problem has been detected in this test
B There are small problems, but no specific look or action is suggested to the participant
C Problems, ISTA indicates there might be things to consider by the laboratory to explain or correct things
BMP Below Minimum of Performance, ISTA indicates that the results were poor and the laboratory need to find explanations and to improve/correct
In-round rating: Presence/absence3 suggested rating systems
1/10 10%2/10 20%3/10 30%4/10 40%
Rate Fixed Number of misclassified samples
A 0 errors
B 1 or 2 errors
C 3 errors
BMP More than 3 errors
Rate Percentage of misclassified samples
A 0% - 5%
B >5% - 10%
C >10% - 20%
BMP >20%
Rate Percentage of misclassified samples
A 0% - 6%
B >6% - 20%
C >20% - 30%
BMP >30%
Example with 10 samples in a Test
System1 System2 System3
2 mistakes 2/10=20%
In-round Rating: Principles
for Quantification
More than ½ of the sample resultsoutside [½ true level; 2 x true level]
C Yes
Probability to observe amore extreme value of the sum
of absolute spiking levels z-scores whenassuming the laboratory provides
accurate resultsis 0.01
No
No
More than 1/6 (~17%) ofthe sample z-scores outside
[-2; +2] (missing sample values are
counted as outside)
BMP Yes
AB Yes No
Too many sample results aretoo far from truth
C Yes The average results by spiking level are not accurate
No
No
There are too manyinaccurate sample
results
AB Yes No
BMP Yes
Qualitative Test (49 labs) Quantitative and Sub-sampling Quantitative Tests (35 labs)
PT06 Rating
0
5
10
15
20
25
30
35
40
45
50
A B C BMP
# o
f L
abs
Rating System 1 Rating System 2 Rating System 3
0
5
10
15
20
25
30
35
40
45
50
A B C BMP
# o
f L
abs
Rating System 1 Rating System 2 Rating System 3
RS1: seed numberRS2: mass of seedRS3: median
% of laboratories which reported an A, B, C or BMP
Qualitative Results Quantitative Results
PT01-PT06: In-round Rating
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
45 50 39 53 59 48
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
45 50 39 53 59 48
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
45 50 39 53 59 48
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
45 50 39 53 59 48
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
0 20 25 50 56 34
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
0 20 25 50 56 34
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
0 20 25 50 56 34
0
20
40
60
80
100
PT
01
PT
02
PT
03
PT
04
PT
05
PT
06
0 20 25 50 56 34
Rating System for 6 PTs based on the in-round rating values
Qualitative Results
Quantitative Results
PT01-PT06: Overall Rating
One testrating
One testScore Value
Overall ratingon six tests
Range onsix tests
# of labs (min. 4 rounds) 6 5 4
A 5 points A 28-30 points 10 10 - -
B 4 points B 27-21points 14 2 12 -C 3 points C 16-20 points 8 - 2 6
BMP 0 points BMP <16 points 2 - - 2
One testrating
One testScore Value
Overall ratingon six tests
Range onsix tests
# of labs (min. 4 rounds) 6 5 4
A 5 points A 28-30 points 0 - - -
B 4 points B 27-21points 5 - 5 -C 3 points C 16-20 points 5 - 1 4
BMP 0 points BMP <16 points 9 - 2 7
4. Performance Data Evaluation
for presence of seed with specified trait(s) in seed lots and specified trait purity
• Laboratory can choose an established and reliable method
• The «Performance Based Approach» is used instead of «one ISTA validated method»(not published in the ISTA Rules)
• Laboratory needs to demonstrate to ISTA accreditation team its ability to perform well with the method routinely used
Performance approved method to be included in lab’s scope of accreditation
• must be defined in terms of method used, the species tested, and the specified trait is to be determined
Performance Data Evaluation
Scope of ISTA Accreditation
Testingprinciple
Attribute/target Primer
Qual./quantspecificity Species
LOD/quant. range [%]
PCR35S-1/35S-235S-F/35S-R
qual. screening 0.03
Rt-PCR35S-F/35S-RLec R/Lec-F
quant. screening 0.1 - 3.035S Promotor Glycine max
Evaluation of Performance Data for each combination species, test method and trait!
For some combinations a PD has to be performed (on seeds, defined number of samples, defined spiking levels)
For other combinations suitable alternative Performance Data may be used(any other data providing effidence that accuracy and repeatability criteria are met)
Performance Data Evaluation
Relevant PD Documents
Accuracy: departure from true valueRepeatability: variability of results by level
Performance Data Evaluation
7
Audit
• Auditors assess whether implementation aspects are appropriately addressed and whether practice complies with the documented procedures
• To that effect, training, equipment management, facilities, reporting and running the overall quality managements system will be assessed
• A complete practical demonstration of any performance approved method will not be required
5. ISTA Workshops on GM Seed Testing
period workshop title location
2002 – 2006
ISTA/FAO Training Course on Varietal Verification and GMO Detection (8)
Argentina, South Africa, Thailand, Slovenia, Egypt, China, Jamaica, Turkey
2004 – 2005
ISTA Workshop on Statistical Aspects of GMO Detection (3)
France, USA, Argentina
2003
ISTA Variety Testing Workshop - Way to the future
ISTA Open Forum Discussion on (Semi)-Quantitative GMO Testing
Switzerland