Valutazione dell’incertezza di misura - INRiM · • ILAC-G8:03/2009, Guidelines on the reporting...

39
nome, date e luogo della conferenza Valutazione dell’incertezza di misura Francesca Pennecchi (INRIM) Formazione: il controllo dei metodi (9/11/2017)

Transcript of Valutazione dell’incertezza di misura - INRiM · • ILAC-G8:03/2009, Guidelines on the reporting...

nome date e luogo della conferenza

Valutazione dellrsquoincertezza di misura

Francesca Pennecchi (INRIM)

Formazione il controllo dei metodi (9112017)

nome date e luogo della conferenza

The GUMThe Guide to the expression of uncertainty in measurement (GUM) was published in 1993 by BIPM IEC IFCC ISO IUPAC IUPAP and OIMLThe organizations who supported the development of this Guide are

- BIPM Bureau International des Poids et Mesures

- IEC International Electrotechnical Commission

- ILAC International Laboratory Accreditation Cooperation(officially joined the seven founding international organizations in 2005)

- IFCC International Federation of Clinical Chemistry(now International Federation of Clinical Chemistry and Laboratory Medicine)

- ISO International Organization for Standardization

- IUPAC International Union of Pure and Applied Chemistry

- IUPAP International Union of Pure and Applied Physics

- OlML International Organization of Legal Metrology 2

nome date e luogo della conferenza

JCGMhttpwwwbipmorgencommitteesjcjcgm

3

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

Recently the Working Group 1 of the Joint Committee for Guides in Metrology (JCGM-WG1) also published two supplements to the GUM and it is currently developing a number of new documents in order to

overcome some of the limitations intrinsic in the GUM approach4

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

5

Relevant UNI standards

30 risultati con laquoincertezzaraquo nel titolo

6

bull UNI CEI 70098-32016 Incertezza di misura - Parte 3 Guida allespressione dellincertezza di misura (recepisce la ISOIEC 98-32008)bull UNI CEI 700992008 Vocabolario Internazionale di Metrologia - Concetti fondamentali e generali e termini correlati (VIM) (recepisce la ISOIEC 992007)bull ISOTS 217482010 Guide to the use of repeatability reproducibility and trueness estimates in measurement uncertainty estimationbull ISOTS 217492005 Measurement uncertainty for metrological applications -- Repeated measurements and nested experiments

bull S L R Ellison A Williams (Eds) Quantifying uncertainty in analytical measurement 3rd Edition EurachemCITAC 2012 ISBN 978-0-948926-30-3 (wwweurachemorg)bull S L R Ellison A Williams Use of uncertainty information in compliance assessment EurachemCITAC 2007 (wwweurachemorg)bull ILAC-G8032009 Guidelines on the reporting of compliance with specification (wwwilacorg)bull ILAC-G172002 Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISOIEC 17025 (wwwilacorg)bull B Magnusson T Naumlykki H Hovind M Krysell Handbook for calculation of measurement uncertainty in environmental laboratories Nordtest Report TR 537 ed 312012 (wwwnordtestinfo)bull EA-416 EA guidelines on the expression of uncertainty in quantitative testing 2003 (wwweuropean-accreditationorg)bull EA-402 M Expression of the uncertainty of measurement in calibration 2013 (wwweuropean-accreditationorg)bull N Majcen P Taylor T Martišius A Menditto M Patriarca Practical examples on traceability measurement uncertainty and validation in chemistry Vol 2 2011 European Commission Joint Research Centre (httpsbookshopeuropaeuenhome)

bull B Magnusson and U Oumlrnemark (eds) Eurachem Guide The Fitness for Purpose of Analytical Methods ndash A Laboratory Guide to Method Validation and Related Topics (2nd ed 2014) ISBN 978-91-87461-59-0 Tradotta in italiano da E Gregori M Patriarca e M Sega (httpswwweurachemorgimagesstoriesGuidespdfMV_guide_2nd_ed_ITpdf)

Relevant standards and guides

7

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

nome date e luogo della conferenza

The GUMThe Guide to the expression of uncertainty in measurement (GUM) was published in 1993 by BIPM IEC IFCC ISO IUPAC IUPAP and OIMLThe organizations who supported the development of this Guide are

- BIPM Bureau International des Poids et Mesures

- IEC International Electrotechnical Commission

- ILAC International Laboratory Accreditation Cooperation(officially joined the seven founding international organizations in 2005)

- IFCC International Federation of Clinical Chemistry(now International Federation of Clinical Chemistry and Laboratory Medicine)

- ISO International Organization for Standardization

- IUPAC International Union of Pure and Applied Chemistry

- IUPAP International Union of Pure and Applied Physics

- OlML International Organization of Legal Metrology 2

nome date e luogo della conferenza

JCGMhttpwwwbipmorgencommitteesjcjcgm

3

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

Recently the Working Group 1 of the Joint Committee for Guides in Metrology (JCGM-WG1) also published two supplements to the GUM and it is currently developing a number of new documents in order to

overcome some of the limitations intrinsic in the GUM approach4

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

5

Relevant UNI standards

30 risultati con laquoincertezzaraquo nel titolo

6

bull UNI CEI 70098-32016 Incertezza di misura - Parte 3 Guida allespressione dellincertezza di misura (recepisce la ISOIEC 98-32008)bull UNI CEI 700992008 Vocabolario Internazionale di Metrologia - Concetti fondamentali e generali e termini correlati (VIM) (recepisce la ISOIEC 992007)bull ISOTS 217482010 Guide to the use of repeatability reproducibility and trueness estimates in measurement uncertainty estimationbull ISOTS 217492005 Measurement uncertainty for metrological applications -- Repeated measurements and nested experiments

bull S L R Ellison A Williams (Eds) Quantifying uncertainty in analytical measurement 3rd Edition EurachemCITAC 2012 ISBN 978-0-948926-30-3 (wwweurachemorg)bull S L R Ellison A Williams Use of uncertainty information in compliance assessment EurachemCITAC 2007 (wwweurachemorg)bull ILAC-G8032009 Guidelines on the reporting of compliance with specification (wwwilacorg)bull ILAC-G172002 Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISOIEC 17025 (wwwilacorg)bull B Magnusson T Naumlykki H Hovind M Krysell Handbook for calculation of measurement uncertainty in environmental laboratories Nordtest Report TR 537 ed 312012 (wwwnordtestinfo)bull EA-416 EA guidelines on the expression of uncertainty in quantitative testing 2003 (wwweuropean-accreditationorg)bull EA-402 M Expression of the uncertainty of measurement in calibration 2013 (wwweuropean-accreditationorg)bull N Majcen P Taylor T Martišius A Menditto M Patriarca Practical examples on traceability measurement uncertainty and validation in chemistry Vol 2 2011 European Commission Joint Research Centre (httpsbookshopeuropaeuenhome)

bull B Magnusson and U Oumlrnemark (eds) Eurachem Guide The Fitness for Purpose of Analytical Methods ndash A Laboratory Guide to Method Validation and Related Topics (2nd ed 2014) ISBN 978-91-87461-59-0 Tradotta in italiano da E Gregori M Patriarca e M Sega (httpswwweurachemorgimagesstoriesGuidespdfMV_guide_2nd_ed_ITpdf)

Relevant standards and guides

7

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

nome date e luogo della conferenza

JCGMhttpwwwbipmorgencommitteesjcjcgm

3

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

Recently the Working Group 1 of the Joint Committee for Guides in Metrology (JCGM-WG1) also published two supplements to the GUM and it is currently developing a number of new documents in order to

overcome some of the limitations intrinsic in the GUM approach4

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

5

Relevant UNI standards

30 risultati con laquoincertezzaraquo nel titolo

6

bull UNI CEI 70098-32016 Incertezza di misura - Parte 3 Guida allespressione dellincertezza di misura (recepisce la ISOIEC 98-32008)bull UNI CEI 700992008 Vocabolario Internazionale di Metrologia - Concetti fondamentali e generali e termini correlati (VIM) (recepisce la ISOIEC 992007)bull ISOTS 217482010 Guide to the use of repeatability reproducibility and trueness estimates in measurement uncertainty estimationbull ISOTS 217492005 Measurement uncertainty for metrological applications -- Repeated measurements and nested experiments

bull S L R Ellison A Williams (Eds) Quantifying uncertainty in analytical measurement 3rd Edition EurachemCITAC 2012 ISBN 978-0-948926-30-3 (wwweurachemorg)bull S L R Ellison A Williams Use of uncertainty information in compliance assessment EurachemCITAC 2007 (wwweurachemorg)bull ILAC-G8032009 Guidelines on the reporting of compliance with specification (wwwilacorg)bull ILAC-G172002 Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISOIEC 17025 (wwwilacorg)bull B Magnusson T Naumlykki H Hovind M Krysell Handbook for calculation of measurement uncertainty in environmental laboratories Nordtest Report TR 537 ed 312012 (wwwnordtestinfo)bull EA-416 EA guidelines on the expression of uncertainty in quantitative testing 2003 (wwweuropean-accreditationorg)bull EA-402 M Expression of the uncertainty of measurement in calibration 2013 (wwweuropean-accreditationorg)bull N Majcen P Taylor T Martišius A Menditto M Patriarca Practical examples on traceability measurement uncertainty and validation in chemistry Vol 2 2011 European Commission Joint Research Centre (httpsbookshopeuropaeuenhome)

bull B Magnusson and U Oumlrnemark (eds) Eurachem Guide The Fitness for Purpose of Analytical Methods ndash A Laboratory Guide to Method Validation and Related Topics (2nd ed 2014) ISBN 978-91-87461-59-0 Tradotta in italiano da E Gregori M Patriarca e M Sega (httpswwweurachemorgimagesstoriesGuidespdfMV_guide_2nd_ed_ITpdf)

Relevant standards and guides

7

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

Recently the Working Group 1 of the Joint Committee for Guides in Metrology (JCGM-WG1) also published two supplements to the GUM and it is currently developing a number of new documents in order to

overcome some of the limitations intrinsic in the GUM approach4

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

5

Relevant UNI standards

30 risultati con laquoincertezzaraquo nel titolo

6

bull UNI CEI 70098-32016 Incertezza di misura - Parte 3 Guida allespressione dellincertezza di misura (recepisce la ISOIEC 98-32008)bull UNI CEI 700992008 Vocabolario Internazionale di Metrologia - Concetti fondamentali e generali e termini correlati (VIM) (recepisce la ISOIEC 992007)bull ISOTS 217482010 Guide to the use of repeatability reproducibility and trueness estimates in measurement uncertainty estimationbull ISOTS 217492005 Measurement uncertainty for metrological applications -- Repeated measurements and nested experiments

bull S L R Ellison A Williams (Eds) Quantifying uncertainty in analytical measurement 3rd Edition EurachemCITAC 2012 ISBN 978-0-948926-30-3 (wwweurachemorg)bull S L R Ellison A Williams Use of uncertainty information in compliance assessment EurachemCITAC 2007 (wwweurachemorg)bull ILAC-G8032009 Guidelines on the reporting of compliance with specification (wwwilacorg)bull ILAC-G172002 Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISOIEC 17025 (wwwilacorg)bull B Magnusson T Naumlykki H Hovind M Krysell Handbook for calculation of measurement uncertainty in environmental laboratories Nordtest Report TR 537 ed 312012 (wwwnordtestinfo)bull EA-416 EA guidelines on the expression of uncertainty in quantitative testing 2003 (wwweuropean-accreditationorg)bull EA-402 M Expression of the uncertainty of measurement in calibration 2013 (wwweuropean-accreditationorg)bull N Majcen P Taylor T Martišius A Menditto M Patriarca Practical examples on traceability measurement uncertainty and validation in chemistry Vol 2 2011 European Commission Joint Research Centre (httpsbookshopeuropaeuenhome)

bull B Magnusson and U Oumlrnemark (eds) Eurachem Guide The Fitness for Purpose of Analytical Methods ndash A Laboratory Guide to Method Validation and Related Topics (2nd ed 2014) ISBN 978-91-87461-59-0 Tradotta in italiano da E Gregori M Patriarca e M Sega (httpswwweurachemorgimagesstoriesGuidespdfMV_guide_2nd_ed_ITpdf)

Relevant standards and guides

7

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

nome date e luogo della conferenza

httpwwwbipmorgenpublicationsguides

5

Relevant UNI standards

30 risultati con laquoincertezzaraquo nel titolo

6

bull UNI CEI 70098-32016 Incertezza di misura - Parte 3 Guida allespressione dellincertezza di misura (recepisce la ISOIEC 98-32008)bull UNI CEI 700992008 Vocabolario Internazionale di Metrologia - Concetti fondamentali e generali e termini correlati (VIM) (recepisce la ISOIEC 992007)bull ISOTS 217482010 Guide to the use of repeatability reproducibility and trueness estimates in measurement uncertainty estimationbull ISOTS 217492005 Measurement uncertainty for metrological applications -- Repeated measurements and nested experiments

bull S L R Ellison A Williams (Eds) Quantifying uncertainty in analytical measurement 3rd Edition EurachemCITAC 2012 ISBN 978-0-948926-30-3 (wwweurachemorg)bull S L R Ellison A Williams Use of uncertainty information in compliance assessment EurachemCITAC 2007 (wwweurachemorg)bull ILAC-G8032009 Guidelines on the reporting of compliance with specification (wwwilacorg)bull ILAC-G172002 Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISOIEC 17025 (wwwilacorg)bull B Magnusson T Naumlykki H Hovind M Krysell Handbook for calculation of measurement uncertainty in environmental laboratories Nordtest Report TR 537 ed 312012 (wwwnordtestinfo)bull EA-416 EA guidelines on the expression of uncertainty in quantitative testing 2003 (wwweuropean-accreditationorg)bull EA-402 M Expression of the uncertainty of measurement in calibration 2013 (wwweuropean-accreditationorg)bull N Majcen P Taylor T Martišius A Menditto M Patriarca Practical examples on traceability measurement uncertainty and validation in chemistry Vol 2 2011 European Commission Joint Research Centre (httpsbookshopeuropaeuenhome)

bull B Magnusson and U Oumlrnemark (eds) Eurachem Guide The Fitness for Purpose of Analytical Methods ndash A Laboratory Guide to Method Validation and Related Topics (2nd ed 2014) ISBN 978-91-87461-59-0 Tradotta in italiano da E Gregori M Patriarca e M Sega (httpswwweurachemorgimagesstoriesGuidespdfMV_guide_2nd_ed_ITpdf)

Relevant standards and guides

7

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

Relevant UNI standards

30 risultati con laquoincertezzaraquo nel titolo

6

bull UNI CEI 70098-32016 Incertezza di misura - Parte 3 Guida allespressione dellincertezza di misura (recepisce la ISOIEC 98-32008)bull UNI CEI 700992008 Vocabolario Internazionale di Metrologia - Concetti fondamentali e generali e termini correlati (VIM) (recepisce la ISOIEC 992007)bull ISOTS 217482010 Guide to the use of repeatability reproducibility and trueness estimates in measurement uncertainty estimationbull ISOTS 217492005 Measurement uncertainty for metrological applications -- Repeated measurements and nested experiments

bull S L R Ellison A Williams (Eds) Quantifying uncertainty in analytical measurement 3rd Edition EurachemCITAC 2012 ISBN 978-0-948926-30-3 (wwweurachemorg)bull S L R Ellison A Williams Use of uncertainty information in compliance assessment EurachemCITAC 2007 (wwweurachemorg)bull ILAC-G8032009 Guidelines on the reporting of compliance with specification (wwwilacorg)bull ILAC-G172002 Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISOIEC 17025 (wwwilacorg)bull B Magnusson T Naumlykki H Hovind M Krysell Handbook for calculation of measurement uncertainty in environmental laboratories Nordtest Report TR 537 ed 312012 (wwwnordtestinfo)bull EA-416 EA guidelines on the expression of uncertainty in quantitative testing 2003 (wwweuropean-accreditationorg)bull EA-402 M Expression of the uncertainty of measurement in calibration 2013 (wwweuropean-accreditationorg)bull N Majcen P Taylor T Martišius A Menditto M Patriarca Practical examples on traceability measurement uncertainty and validation in chemistry Vol 2 2011 European Commission Joint Research Centre (httpsbookshopeuropaeuenhome)

bull B Magnusson and U Oumlrnemark (eds) Eurachem Guide The Fitness for Purpose of Analytical Methods ndash A Laboratory Guide to Method Validation and Related Topics (2nd ed 2014) ISBN 978-91-87461-59-0 Tradotta in italiano da E Gregori M Patriarca e M Sega (httpswwweurachemorgimagesstoriesGuidespdfMV_guide_2nd_ed_ITpdf)

Relevant standards and guides

7

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

bull UNI CEI 70098-32016 Incertezza di misura - Parte 3 Guida allespressione dellincertezza di misura (recepisce la ISOIEC 98-32008)bull UNI CEI 700992008 Vocabolario Internazionale di Metrologia - Concetti fondamentali e generali e termini correlati (VIM) (recepisce la ISOIEC 992007)bull ISOTS 217482010 Guide to the use of repeatability reproducibility and trueness estimates in measurement uncertainty estimationbull ISOTS 217492005 Measurement uncertainty for metrological applications -- Repeated measurements and nested experiments

bull S L R Ellison A Williams (Eds) Quantifying uncertainty in analytical measurement 3rd Edition EurachemCITAC 2012 ISBN 978-0-948926-30-3 (wwweurachemorg)bull S L R Ellison A Williams Use of uncertainty information in compliance assessment EurachemCITAC 2007 (wwweurachemorg)bull ILAC-G8032009 Guidelines on the reporting of compliance with specification (wwwilacorg)bull ILAC-G172002 Introducing the concept of uncertainty of measurement in testing in association with the application of the standard ISOIEC 17025 (wwwilacorg)bull B Magnusson T Naumlykki H Hovind M Krysell Handbook for calculation of measurement uncertainty in environmental laboratories Nordtest Report TR 537 ed 312012 (wwwnordtestinfo)bull EA-416 EA guidelines on the expression of uncertainty in quantitative testing 2003 (wwweuropean-accreditationorg)bull EA-402 M Expression of the uncertainty of measurement in calibration 2013 (wwweuropean-accreditationorg)bull N Majcen P Taylor T Martišius A Menditto M Patriarca Practical examples on traceability measurement uncertainty and validation in chemistry Vol 2 2011 European Commission Joint Research Centre (httpsbookshopeuropaeuenhome)

bull B Magnusson and U Oumlrnemark (eds) Eurachem Guide The Fitness for Purpose of Analytical Methods ndash A Laboratory Guide to Method Validation and Related Topics (2nd ed 2014) ISBN 978-91-87461-59-0 Tradotta in italiano da E Gregori M Patriarca e M Sega (httpswwweurachemorgimagesstoriesGuidespdfMV_guide_2nd_ed_ITpdf)

Relevant standards and guides

7

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

8

Scope

- General rules for evaluating and expressing uncertainty inmeasurement at various levels of accuracy and in many fields mdash fromthe shop floor to fundamental research

- Principles applicable to a broad spectrum of measurements

- Expression of uncertainty in the measurement of a well-definedphysical quantity mdash the measurand mdash that can be characterized by anessentially unique value

The GUM (JCGM 1002008)

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

9

311 The objective of a measurement is to determine the value of the measurand that is the value of the particular to be measuredA measurement therefore begins with an appropriate specification of the measurand the method of measurement and the measurement procedure

Definitions concepts metrological terms

Set of operations having the object of determining a value of a quantity

Magnitude of a particular quantity expressed as a unit of measurement multiplied by a number EXAMPLE Mass of a body 0152 kg or 152 g

From the International vocabulary of basic and general terms in metrology (VIM)

Particular quantity subject to measurementEXAMPLE Vapour pressure of a given sample of water at 20 degC

Logical sequence of operations described generically used in the performance of measurements

Set of operations described specifically used in the performance of particular measurements according to a given method

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

10

B211result of a measurementvalue attributed to a measurand obtained by measurement

NOTE 2 A complete statement of the result of a measurement includes information about the uncertainty of measurement

B218uncertainty (of measurement)parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand

It is an estimate of the value of the measurand

It indicates the degree of reliability in the result

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

11

315 Variations in repeated observations are assumed to arise becauseinfluence quantities that can affect the measurement result are not heldcompletely constant

316 The mathematical model of the measurement that transforms the set ofrepeated observations into the measurement result is of critical importancebecause in addition to the observations it generally includes various influencequantities that are inexactly known This lack of knowledge contributes to theuncertainty of the measurement result as do the variations of the repeatedobservations and any uncertainty associated with the mathematical model itself

quantity that is not the measurand but that affects the result of the measurement

Random errors arise from unpredictable or stochastic temporal and spatial variations of influence quantities random effects give rise to variations in repeated observations of the measurand

Systematic errors likerandom errors cannot beeliminated but they canoften be reduced by acorrection or correctionfactor

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

12

B214accuracy of measurementcloseness of the agreement between the result of a measurement and a true value of the measurandhellipNOTE 2 The term precision should not be used for ldquoaccuracyrdquo

215 [JCGM 2002012 International vocabulary of metrology ndash Basic and general concepts and associated terms (VIM)]measurement precisioncloseness of agreement between indications or measured quantity values obtained by replicate measurements on the same or similar objects under specified conditionshellipNOTE 2 The lsquospecified conditionsrsquo can be for examplerepeatability conditions of measurement hellip orreproducibility conditions of measurement hellip

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

13

B215repeatability (of results of measurements)

closeness of the agreement between the results of successive measurements of the same measurand carried out under the same conditions of measurementNOTE 1 These conditions are called repeatability conditionsNOTE 2 Repeatability conditions includemdash the same measurement proceduremdash the same observermdash the same measuring instrument used under the same conditionsmdash the same locationmdash repetition over a short period of time

B216reproducibility (of results of measurements)closeness of the agreement between the results of measurements of the same measurand carried out under changed conditions of measurementNOTE 1 A valid statement of reproducibility requires specification of the conditions changedNOTE 2 The changed conditions may includemdash principle of measurementmdash method of measurementmdash observermdash measuring instrumentmdash reference standardmdash locationmdash conditions of usemdash time

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

14

332 In practice there are many possible sources of uncertainty in a measurement including

a) incomplete definition of the measurandb) imperfect reaIization of the definition of the measurandc) nonrepresentative sampling mdash the sample measured may not represent the defined measurandd) inadequate knowledge of the effects of environmental conditions on the measurement or imperfect measurement of environmental conditionse) personal bias in reading analogue instrumentsf) finite instrument resolution or discrimination thresholdg) inexact values of measurement standards and reference materialsh) inexact values of constants and other parameters obtained from external sources and used in the data-reduction algorithmi) approximations and assumptions incorporated in the measurement method and procedurej) variations in repeated observations of the measurand under apparently identical conditions

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

15

ERRORS

SystematicRandom

Type A Type B

UNCERTAINTIES

Example variations in the influence quantities

Example drift of a standard or of an instrument

Solution apply a correction or a correction factor

Evaluation statistical analysis of series of observations ie by means of sample standard deviations

Evaluation standard deviation of assumed probability distributions based on experience or other information

Solution increase the number of measurements

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

16

334 The purpose of the Type A and Type B classification is to indicate the twodifferent ways of evaluating uncertainty components and is for convenienceof discussion only the classification is not meant to indicate that there is anydifference in the nature of the components resulting from the two types ofevaluation Both types of evaluation are based on probability distributionsand the uncertainty components resulting from either type are quantified byvariances or standard deviations

NXXXfY 21

Nxxxfy 21

Measurement model

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

17

Evaluation of the input standard uncertaintiesType A when n repeated independent observations Xik of Xiare available

n

kkii x

nx

1

1

n

kikii xx

nnxu

1

2

2 )()1(

11)(

Estimate of Xi

Experimental varianceof the mean xi

)()( 2ii xuxu Type A standard uncertainty associated with xi

1 ni Degrees of freedom for u(xi)

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

18

)( i

i

xux

2

)()(

21

i

ii xu

xu

Evaluation of the input standard uncertaintiesType B whenever repeated observations Xik of Xi are NOT available

Estimate of Xi

Type B standard uncertainty associated with xi

Degrees of freedom for u(xi)

Type B standard uncertainty is evaluated by scientific judgement based on all of the available information on the possible variability of Xi such as

previous measurement dataexperience with or general knowledge of the behaviour and properties of relevant

materials and instrumentsmanufacturers specificationsdata provided in calibration and other certificatesuncertainties assigned to reference data taken from handbooks

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

19

Type B cont

Examples

An uncertainty defines an interval I having 95 level of confidence normal distributionu = I 196

A quantity value lies within the interval a- to a+ rectangular distributionu = (a+ - a-)radic12

A quantity value cycles sinusoidally within the interval a- to a+ U-shaped (arcsine) distribution u = (a+ - a-)radic8

682 coverage interval

577 coverage interval

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

20

yukU cExpanded uncertainty

AIM 1 calculate the uncertainty associated with y

AIM 2 calculate the expanded uncertainty associated with y

Nxxxfy 21

1xu 2xu Nxu

Input standard uncertainties

yuc

Output standard uncertainty

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

21

)( )( 22

1

2i

N

i ic xu

xfyu )( 2

1

1 1ji

N

i

N

ij ji

xxuxf

xf

)( )( )( 21

1 1jiji

N

i

N

ij ji

xuxuxxrxf

xf

OR

Solution to AIM 1 Law of propagation of uncertainty (LPU)

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

22

Where

)( ixu

ixf

)( ji xxu

)()()(

)(ji

jiji xuxu

xxuxxr

Standard uncertainty associated with xi

Sensitivity coefficient it describes how the output estimate y varies with changes in thevalues of the input estimate xi

(Estimated) covariance between Xi and Xj

(Estimated) correlation coefficientbetween Xi and Xj

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

23

Solution to AIM 2 multiply the combined standard uncertainty uc(y) by a coverage factor k

- Interval [y minus U y + U] is a coverage interval at a desired coverage probability p ie it is expected to encompass a large fraction of the distribution of values that could reasonably be attributed to the measurand

- Coverage factor k depends on p (ie 95 or 99 ) and on the form of the probability distribution modelling the measurement result

yukU c

How to choose k

bull Y can be approximated by a normal distribution if- input ui are all of the same order of magnitude- no type A unc with few degrees of freedom dominating- no type B unc resulting from a rectangular distribution

dominating- the number N if input quantities is hight

bull Y can be approximated by a a (Studentrsquos) t-distribution with effective degrees of freedom eff provided by the Welch-Satterthwaite formula

)(

)(

1

4

4

N

i i

i

ceff yu

yu

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

24

Simple example a length measurement

TLL m 201 20

bullL20 is the measurand ie the length at 20 degC of an artifact

bullLm instrumental observationindication

bullα coefficient of thermal expansion of the artifact being measured

bull(20-T) deviation in temperature from the 20 degC reference temperature

Measurement model

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

25

Available information on the input quantities

bull n = 5 independent repeated observations of Lm from whichsample mean lm = 800 mm and sample standard deviation s = 10 mmType A evaluation u(lm) = sradicn = 10radic5 = 447 mm with 1 = 4 degrees of freedom

bull Plausible values of the thermal coefficient are encompassed in the interval [23middot10-6 27middot10-6]degC-1 hence α = (27+23)middot10-62 = 25 middot10-6 degC-1 Type B evaluation assuming a uniform (rectangular) distributionR(23middot10-6 27middot10-6) u(α) = (27-23)middot10-6radic12 = 115 middot10-6 degC-1 with 2 = 50 degrees of freedom

bull Temperature at the time of the individual observations was not recorded The range of temperature variation is [17 33]degC and it is said to represent the amplitude of an approximately cyclical variation in time of the temperature under a thermostatic system hence t = (17+33)2 = 25degC Type B evaluation assuming a U-shaped distribution u(t) = (33-17)(2radic2) = 566degC Let us suppose that the value of u(t) is reliable to about 25 This may be taken to mean that the relative uncertainty of the uncertainty is ∆u(xi)u(xi) = 025 hence 3 = (025)minus22 = 8

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

26

Measurand estimate

mm 9799252010251 800 620 l

1-20

20

20

20

Cmm 002

Cmm 1520020

0999875201

201 model From

m

m

m

m

LT

L

TLL

TLL

TLLLPU

mm 474)(

mm 98919mm 00128 00003 975919)()( 2222

1

2

yu

xuxfyu

c

i

N

i ic

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

27

Coverage interval at 95 level of confidence

40044

8000016

50109

40366993

39956)(

)(8

1

4

4

N

i i

i

ceff yu

yu

mm ]33812 47787[][

mm 4312mm 474782 )(

9502095020

20eff950950

UlUllutU c

NB If we were using the Normal approximation (k = 196) we would have been a much narrower interval

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

28

There are situations where the GUM uncertainty framework might not be satisfactory including those where

a) the measurement function is highly non-linearb) the probability distributions for the input quantities are asymmetricc) the uncertainty contributions |c1|u(x1) hellip |cN|u(xN) are not of approximately the same magnituded) the probability distribution for the output quantity is either asymmetric or not a Gaussian or a scaled-and-shifted t-distributione) the partial derivatives (or numerical approximations to them) can be difficult to be provided for complicated measurement modelsf) the input quantities are not independentg) there is more than one measurand

Limitations to the GUM applicability

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

29

JCGM 1012008 - Numerical Methods for the Propagation of Distributions

ldquohellipThis Supplement is concerned with the concept of the propagation of probability distributions through a model of measurement as a basis for the evaluation of uncertainty of measurement and its implementation by Monte Carlo simulationhellip In particular the provision of the probability density function for the output quantity value permits the determination of a coverageinterval for that value corresponding to a prescribed coverageprobabilityhelliprdquo

NXXXfY 21

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

30

JCGM 1022011 - The treatment of models with more than one output quantity

Nmm

N

N

xxxfy

xxxfyxxxfy

21

2122

2111

ldquohellipThe Guide to the Expression of

Uncertainty in Measurement deals mainlywith the case of one measurand (hellipthe univariate or scalar case) However thereexist several experimental situations in whichmore than one measurand are determinedsimultaneously from a common set of input quantities (hellipmultivariate or vectorcases) In these cases uncertainty propagation treated in Clause 5 of the GUM needs appropriate extensionhelliprdquo

ji

N

i

N

j j

h

i

khk xx

xy

xyyy covcov

1 1

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

31

LPU vs MC method an example on

M Sega F Pennecchi S Rinaldi F Rolle Analytica Chimica Acta Volume 920 pp 10 ndash 17 (2016)

ldquoUncertainty evaluation for the quantification of low masses of benzo[a]pyrene Comparison between the Law of Propagation of Uncertainty and the Monte Carlo methodrdquo

Use of an isotopically labelled compound as internal standard (IS) having the same characteristics of the analyte

ISc

cIS

mAmAf

ISE

ISEEE A

mAfm

The mass of BaP in the sample extracts is calculated according to

A response factor f is calculated from the chromatographic areas of BaP and IS (obtained as mean values of repeated analyses of the calibration solutions) and from their masses according to

EN 15594 ndash ldquoAir Quality ndash Standard method for the measurement of concentration of benzo[a]pyrene in ambient airrdquo (2008)

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

32

LPU vs MC method cont

Model equationISE

ISEEE A

mAfm

Uncertaintycomponentu(xi)

Uncertainty sourceStandard uncertainty

valueu(xi)

δmEδxiContribution to

u(mE) |δmEδxi|u(xi)

u(AISEmean)Peak area of the

internal std 42E+05 ua -14E-07 ng (ua)-1 60E-02 ng

u(f) BaP calibration factor 17E-02 67E-01 ng 11E-02 ng

u(AEmean) Peak area of BaP 10E+06 ua 53E-08 ng (ua)-1 55E-02 ng

u(mISE) Mass of internal std 36E-03 ng 16 60E-03 ng

cov(xi xj) δmEδxi δmEδxjContribution to

u2(mE)

cov(AISE AE) Covariance AISE and AE 43 E+11(ua)2 -77E-15 ng2 (ua)-2 -66E-03 ng2

cov(mISE f) Covariance mISE and f -32E-05 ng 11 ng -71E-05 ng2

uc(mE) = 0012 ng U(mE)= k095 uc(mE) = 0035 (k095 = 28) U(mE)rel = 86

Uncertainty budget for the mass of BaP in the sample obtained after the first extraction (high level)mE = 0405 ng

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

33

LPU vs MC method cont

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

34

The NIST Uncertainty Machinehttpsuncertaintynistgov

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

35

The NPL softwareshttpwwwnplcoukscience-technologymathematics-modelling-and-simulationproducts-and-servicesIn laquoSoftware Downloadsraquo

-10 -8 -6 -4 -2 0 2 4 6 8 10

0

002

004

006

008

01

012

014

016

018

02

Probability distribution for the output quantity

Value

Pro

babi

lity

dens

ity

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

36

The LNE-MCM 11 softwarehttpwwwlnefrfrlogicielsMCMlogiciel-lne-mcmasp

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

37

The MSL - MST software

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

httpswwwmslirlcrinzservicesspecialist-user-groupsmeasurement-software-toolkitmst-software

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

38

The GUM Workbenchhttpwwwmetrodatadeindex_enhtml

PhD Course on ldquoThe evaluation of uncertainty in measurementrdquo ndash Cov Int amp Suppl 1

Thanks for the attention

39

Thanks for the attention

39