12.Statistics GREY

7/29/2019 12.Statistics GREY

http://slidepdf.com/reader/full/12statistics-grey 1/47

Unit 4

Statistics,

Detection Limits

and UncertaintyExperts Teaching from Practical Experience



© Kinectrics I nc. 2012 2

Unit 4 – Topics

Statistical Analysis

Detection Limits

Decision thresholds & detection levels

Instrument Detection Limits vs. Method Detection

Limits




Unit 4 – Topics

Uncertainty Requirements

Guide to Uncertainty in Measurement (GUM)

Uncertainty budgets

Sampling uncertainty




Unit 4 – Learning Objectives

Describe the responsibilities of the user of the

standards with regard to Detection Limits

Identify the guidance on analysis of samples and

reporting of results

Describe the changes in the guidance onestimating and reporting uncertainties



Unit 4a

Statistics

Experts Teaching from Practical Experience




MARSSIM (Chapter 8) recommends the followingsequence of steps:

1. Review the Data Quality Objectives (DQOs) and

Sampling Design

2. Conduct a Preliminary Data Review

3. Select the Tests

4. Verify the Assumptions of the Tests

5. Draw Conclusions from the Data

More detail is given in EPA QA/G-9S

Interpretation of Survey Results


http://slidepdf.com/reader/full/12statistics-grey 7/47© Kinectrics I nc. 2012 7

Review the objectives of the study

Translate the objectives into statistical hypotheses

Translate the objectives into limits on Type I & Type

II errors Review the sampling design & note any special

features or potential problems

Review the DQOs and Sampling Design



Review Quality Assurance reports• Look for problems or anomalies

Calculate basic statistical quantities

• Calculate percentiles, measures of central tendency,dispersion and, if the data involves two variables, the

correlation coefficient

Graph the data

• Select graphical representations that illuminate the

structure of the data

Conduct a Preliminary Data Review



Select the statistical method• Follow the “Decision Tree” from EPA QA/G-9S

Identify assumptions underlying the test

• List the key underlying assumptions such asdistributional form, dispersion, independence, etc

• Note any sensitive assumptions where relatively small

deviations could jeopardize the validity of the test

Select the Statistical Methods



Determine approach for verifying assumptions• Review (or develop) a statistical model for the data

• Select the methods for verifying the assumptions

Perform tests of assumptions• Adjust for distributional assumptions (if warranted)

• Perform the calculations required for the tests

Determine corrective actions

Verify the Assumptions



Determine corrective actions (if required) • Determine if data transformation will correct the

problem

• If data is missing, explore collecting more data or using

theoretical justification

• Consider robust procedures or nonparametric

hypothesis tests

Verify the Assumptions



Perform the statistical procedure• Perform & document the statistical tests

• Identify outliers and recalculate if necessary

Draw study conclusions• If the null hypothesis is rejected, draw conclusions

& document

• If the null hypothesis is not rejected, verify limits on

decision errors then draw conclusions & document

• Interpret the results

Draw Conclusions from the Data



Evaluate performance of the sampling design• Evaluate the statistical power of the design over the full

range of parameter values

Draw Conclusions from the Data



Unit 4b

Detection Limits




Both N228.4-10 & N2885.-11 caution thatterminology and the definitions used for these

concepts (non-detect level & detection limit) are not

always consistent. It is the responsibility of the user

to understand, document, and justify the detectionlimit reported by any laboratory engaged to perform

analyses.

Definitions



Definitions

Organization or Discipline LC LD

CSA N288.4/5 Non-detect Level (LC) Detection Limit (LD)

Health Physics

(ANSI/HPS N13.30-1996)Critical Level (CL) Lower Limit of Detection (LLD)

Occupational Hygiene

(AIHA, IOHA)Limit of Detection (LOD) Limit of Quantification (LOQ)

Environmental Analytical

Chemistry Cmte. of ACSLimit of Detection (LOD) Limit of Quantification (LOQ)

IUPAC Detection Decisions (LC)Minimum Detectable Value or

Detection Limit (LD)

ISO 11929:2010 Decision Threshold (y*) Detection Limit (y#)



Non-detect level — the level below whichquantitative results are not obtained from the

measurement system or analysis method selected.

• The non-detect level is the smallest value of the

measurand for which the probability of a wrongconclusion that the measurand is present when it

actually is not present (‘error of the first kind’ or a ‘falsepositive error’) does not exceed a specified probability,

α.

Excerpt from Clause 3.1 of CSA N288.4-10 and N288.5-11

Definitions



Detection limit — the level (relative to background)above which an effect can confidently be measured.

• The detection limit is the smallest value of the

measurand for which the probability of a wrong

conclusion that the measurand is not present when itactually is present (‘error of the second kind’ of a ‘falsenegative error’) does not exceed a specifiedprobability, β.

Excerpt from Clause 3.1 of CSA N288.4-10 and N288.5-11

Definitions



Non-detect Level (LC) for Type I Error (α)Detection Limit (LD) for Type II Error (β)

Definitions

From IUPAC Recommendations 1995



Definitions

CSA N288.4-10

Annex D, Table D.1

Formulae for non-detect (critical) level and detection limit



The level below which quantitative results are notobtained from the measurement system or analysis

method selected is called the non-detect level.

• This level shall be defined and its derivation should be

documented.• Measurements below this level are often reported as

being “less than” some value.

Paraphrased from Clause 8.1.10 of CSA N288.5-11

CSA N288.5-11



The treatment of results that are less than the non-detect level for the measurement shall be defined

and documented.

The results of measurements that are below the non-

detect level may be reported as “not detected” or asbeing below a “less than value”.

The values of the non-detect level should be

documented and reported.

Paraphrased from Clause 8.3 of CSA N288.4-10

CSA N288.4-10




Quantitative numerical values should be reportedrather than “less than” some value or non-detect

level.

• The requirement for numerical values needs to be

considered within the context of laboratory capabilitiesand the proximity of the result to any applicable

benchmark value.

Paraphrased from Clause 8.1.10 of CSA N288.5-11

CSA N288.5-11




The user is cautioned that these formula

might not be

appropriate in all

situations,

particularly in low-

level counting.

(see Strom & MacLellan,

Health Physics, 81(1),

2001).

Low Count Rates




A note following Clause 8.1.9 of N288.5-11 warnsthat:

• …some laboratories might report an instrumentdetection level that is often much lower than the

detection level of the method (which includes anyrequired sample preparation). It is the responsibility of

the user to understand, document, and justify the

detection limit reported by any laboratory engaged to

perform analyses.

Both standards generally assume (but often do not

always explicitly state) that the detection level is the

Method Detection Level

Instrument vs. Method Detection Limits




Unit 4b – Summary and Review

Detection Limits

Non-detect level & detection limit

Instrument Detection Limits vs. Method Detection

Limits

Learning Objective

Describe the responsibilities of the user of thestandards with regard to Detection Limits



Unit 4c

Uncertainty





Uncertainty - a quantitative expression of error that

results from incomplete knowledge or informationabout a parameter or value.

– Measurement Uncertainty (from JCGM 200:2012 – International

Vocabulary of Metrology): non-negative parameter characterizing the

dispersion of the quantity values being attributed to a measurand,

based on the information used• Statistical uncertainty - that component of uncertainty

which arises from imprecision.

• Systematic uncertainty - that component of uncertainty

which arises from biases. Excerpt from Clause 3.1 of CSA N288.4-10

• Type A uncertainty - determined by repeated

measurement

• Type B uncertainty - determined in any other manner

Uncertainty




N288.4: The uncertainty associated with each

measured or calculated value should be estimated. Excerpt from Clause 9.3.3.1 of CSA N288.4-10

N288.5: The uncertainty associated with results of

effluent monitoring measurements and any doseestimates derived from them should be discussed in

the report. Excerpt from Clause 9.3.3..2.1 of CSA N288.5-11

Both: The uncertainty should take into account bothsampling and measurement errors. Sampling errors

cannot always be quantified but they shall be kept to

a minimum by design of the monitoring program.

Requirements




Introduction to Uncertainty

Two introductory texts on uncertainty:




The Guide to Uncertainty in Measurement (GUM)

was prepared by the Joint Committee for Guides in

Metrology (JCGM)

• Originally published in 1995 (JCGM 100:1995)

• Minor revision published in 2008 (JCGM 100:2008) – Available for download from the BIPM website at

http://www.bipm.org/en/publications/guides/gum.html

The 1995 version of the GUM was adopted by ISO &

IEC as ISO/IEC Guide 98-3:2008• the 2008 revision has not yet been adopted by

ISO/IEC

Guide to Uncertainty in Measurement




The JCGM Working Group on Uncertainty in

Measurement is preparing additional guidance on

the “Evaluation of Measurement Data”:

• Guide to the expression of uncertainty in measurement

(JCGM 100:2008)• Propagation of distributions using a Monte Carlo

method – Supplement 1 to the GUM (JCGM 101:2008)

• Extension to any number of output quantities –

Supplement 2 to the GUM (JCGM 102:2011)• Introduction to the “Guide to the expression of

uncertainty in measurement” and related documents(JCGM 104:2009)

• Other documents & supplements are in preparation

Evaluation of Measurement Data




UKAS M3003 (Ed. 2, Jan. 2007) “The Expression of Uncertainty and Confidence in Measurement” • Provides an introduction to the subject with examples

of the application of the guidance given in the GUM

• http://www.ukas.com/library/Technical-Information/Pubs-Technical-Articles/Pubs-

List/M3003.pdf

The Eurachem/CITAC Guide (Ramsey & Ellison,

2007) “Measurement of uncertainty arising fromsampling: A guide to methods and approaches”• Provides additional guidance on the estimation of

uncertainty due to sampling and sample preparation

• http://www.eurachem.org/guides/pdf/UfS_2007.pdf

Further Guidance




The American Society for Quality (ASQ) defines an

‘uncertainty budget’ as:

a statement of measurement uncertainty, of the

components of that measurement uncertainty, and of

their calculation and combination A National Research Council template for

Uncertainty Budgets (an Excel spreadsheet) is

available at:

www.nrc-nrc.gc.ca/obj/inms-ienm/doc/clas-clas/uncertainty_budget_template.xls

Uncertainty Budgets




Example: Calculation of activity by liquid

scintillation counting

A = {(C-B) x exp(-λt) x R x S} / {V x T x ε x Pϒ}

Uncertainty Budgets

Symbol Symbol A Activity V Volume

C Gross Counts T Time

B Background Counts ε Efficiency

exp(-λt) Decay Factor Pϒ

Emission Probability

R Random Summing

Correction

S Self-absorption Correction




Uncertainty Budgets

Component Value Uncertainty Distribution Divisor

Relative

Uncertainty

Gross Counts 414 20.3 Normal (1σ) 1 4.91%

Background Counts 22 4.7 Normal (1σ) 1 21.32%

Decay 1 0.10% Normal (1σ) 1 0.10%

R 1 0.30% Normal (1σ) 1 0.30%

S 1 0.30% Normal (1σ) 1 0.30%

Volume (l) 0.001 0.00002 Rectangular (half-range) √3 1.15%

Live Time (s) 600 0.001 Rectangular (half-range) √3 0.00%

Efficiency 0.34 3.50% Normal (2σ) 2 1.75%

Emission Probability 1 0.50% Normal (2σ) 2 0.25%

ACTIVITY (Bq/l) 1922 221 Normal (2σ) 11.49%




P(x) = 1/2a, for (b –a) < x < (b+a)

0, otherwise

E(x) = Mean = bE(x2) = b2 + a2/3

σ2 = E(x2) – E(x)2 = a2/3

σ = a/√3

Rectangular Distribution




Sampling Uncertainty




Sampling Uncertainty should be included but it

may be difficult (or impossible) to quantify.

Possible approaches include:

• Empirical Methods

– Duplicate samples

– Multiple protocols

– Collaborative trial

– Sampling proficiency test

– Reference sampling method/target

• Modeling

– Cause & Effect Modeling

– Gy’s Sampling Theory






Method

Component Estimated

Sampling Analytical

Precision Bias Precision Bias

Duplicate Samples(Single sampler using a single

sampling protocol)

Yes No Yes No *

(may be estimated by

including Certified

Reference Materials)

Multiple Protocols(Single sampler using multiple

sampling protocols)

Between protocols Yes No *

(may be estimated by

including Certified

Reference Materials)

Collaborative Trial(Multiple samplers using the same

sampling protocol)

Between samplers Yes Yes

Sampling Proficiency Test(Multiple samplers each using a

different sampling protocols)

Between samplers &

protocolsYes Yes




“Model uncertainty” is an assessment of the degreeof confidence that a mathematical model is a

"correct“ representation of the system.

Model uncertainty includes:

• An estimate of the uncertainty due to the structure of the model (structure uncertainty); and

• An estimate of uncertainty in each of the parameters

used in the risk assessment equations (parameter

uncertainty).

Model Uncertainty




Estimates of the uncertainties in the N288 models

are generally not available:

• Excerpt from CSA N288.1-08 Clause 4.2.9 –

“Conservatism is introduced into the current model by

selecting conservative values for food, water, soil, andair intake rates for the representative person, typically

at the 95th percentile level.”

Model Uncertainty






• IAEA TRS 472 (Handbook of Parameter Values for the

Prediction of Radionuclide Transfer in Terrestrial and

Freshwater Environments) – “Estimations … of uncertainty about each such value were carried out by

applying statistical analysis, where possible ... In some

cases, the values were given without a statement of

uncertainty or a range, because of the limited data

available.”

Model Uncertainty






• NCRP Report 164 (Uncertainties in Internal Dose

Assessment) – The dose limits that are recommended

by the International Commission on RadiologicalProtection (ICRP) for regulatory purposes are based

on the use of values of dose per unit intake that are to

be applied without any consideration of uncertainty.

Model Uncertainty




The number of significant figures quoted in a result

shall not imply a degree of precision greater than

that warranted by the sources of uncertainty.

• Uncertainty should be rounded to one significant figure

• The least significant figure in the result shouldcorrespond to the significant figure in the uncertainty

(e.g., 1900 ± 200 Bq/l).

• More significant figures should be carried for

calculation steps than what is reported in the finalstage.

Paraphrased from Clause 9.3 of CSA N288.4-10 and CSA N288.5-11

Reporting Uncertainty



Unit 4c – Summary and Review

Uncertainty

ISO Guide to Uncertainty in Measurement (GUM)

Uncertainty budgets

Sampling uncertainty

Reporting uncertainty

Learning Objectives

Identify the guidance on analysis of samples andreporting of results

Describe the changes in the guidance on

estimating and reporting uncertainties

12.Statistics GREY

Documents

Transcript of 12.Statistics GREY