Uncertainty Assessment in Computational Dosimetry: A ...old.enea.it/.../Siebert.pdf · Siebert...

32
1 Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 1 Uncertainty Assessment in Computational Uncertainty Assessment in Computational Dosimetry: Dosimetry: A comparison of Approaches A comparison of Approaches Bernd R.L. Siebert Retired from Physikalisch-Technische Bundesanstalt Bundesallee 100, D 38106 Braunschweig GUM-based Uncertainty in Radiation Protection Survey on Methods and Practices This presentation is a tutorial, not a scientific paper. A presentation is sometimes hard to understand without animation and explanations. Furthermore the proceedings will not be available before March 2008. For these reasons I use the “annotations” part of this file to make up for the missing animation and for providing additional information. The latter could help those who want to apply the methods discussed before March 2008. However, this cannot serve as a substitute for the final paper. Furthermore, in this part at some occasions, I respond to comments, discussions and problems heard of during the meeting. It has been said that “sometimes one would never get a result if one were to wait for the uncertainty evaluation and that there are cases where such an evaluation is not possible”. The answer to this statement is simple, without at least a rough estimate of uncertainty one has little judgement on the reliability of the result given and an uncertainty evaluation may be very difficult in some cases, but it is never impossible. See also Introduction page #3, Note in bottom part. The Acrobat layout should allow to print 2 pages on 1 page.

Transcript of Uncertainty Assessment in Computational Dosimetry: A ...old.enea.it/.../Siebert.pdf · Siebert...

  • 1

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 1

    Uncertainty Assessment in Computational Uncertainty Assessment in Computational Dosimetry:Dosimetry:

    A comparison of ApproachesA comparison of Approaches

    Bernd R.L. Siebert

    Retired from

    Physikalisch-Technische BundesanstaltBundesallee 100, D 38106 Braunschweig

    GUM-based Uncertaintyin Radiation Protection

    Survey on Methods and Practices

    This presentation is a tutorial, not a scientific paper.A presentation is sometimes hard to understand without animation and explanations. Furthermore the proceedings will not be available before March 2008. For these reasons I use the “annotations” part of this file to make up for the missing animation and for providing additional information. The latter could help those who want to apply the methods discussed before March 2008. However, this cannot serve as a substitute for the final paper.Furthermore, in this part at some occasions, I respond to comments, discussions and problems heard of during the meeting.It has been said that “sometimes one would never get a result if one were to wait for the uncertainty evaluation and that there are cases where such an evaluation is not possible”.The answer to this statement is simple, without at least a rough estimate of uncertainty one has little judgement on the reliability of the result given and an uncertainty evaluation may be very difficult in some cases, but it is never impossible.See also Introduction page #3, Note in bottom part.The Acrobat layout should allow to print 2 pages on 1 page.

  • 2

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 2

    Content

    Introduction 2

    Evolution of the GUM (JCGM / WG 1) 3

    “Derivation” of Markov Formula 12

    GUM Framework follows from Markov Formula 14

    Expanded Uncertainty ⇔ Coverage Interval 17

    Analytical and Numerical Methods 18

    A Monte Carlo Method 21

    Conformity 26

    Summary and Conclusion 29

    Main aim:To familiarise with basic conceptsand principles of the GUM and toshow that a Monte Carlo Methodprovides reliable solutions.

    GUM: BIPM, IEC, IFCC, ISO, IUPAC, IUPAP and OIML 1995Guide to the Expression of Uncertainty in Measurement(Geneva, Switzerland: International Organisation for

    Standardisation) ISBN 92-67-10188-9.

    The Introduction is brief because the usual “motivation” is not needed, as you would not participate in this workshop if you were not convinced that stating the uncertainty is necessary.

    Evolution of the GUM: Using a recent publication by W. Bich (chairman of the Joint Commission on Guides for Metrology Working Group 1 (http://www.bipm.org/en/committees/jc/jcgm/wg1.html ) et al. the present viewof the concepts behind the GUM and its future development will be reported.

    The heart piece of this presentation is the section “Derivation” of Markov formula, if one understands this formula one understands GUM as GUM-Framework follows from Markov Formula and the central limit theorem.

    An extra section on Expanded Uncertainty ⇔ Coverage Interval is provided, as these concepts are often misunderstood in practice.

    Analytical and Numerical Methods will be only treated briefly as emphasis is given on using a Monte Carlo Method for evaluating uncertainty.

    A section on Conformity is added since this information may be important for calibration laboratories.

  • 3

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 3

    INTRODUCTION

    Why uncertainty and why GUM ?

    GUM is useful in many cases

    GUM versus error analysis

    Incomplete knowledge and probability density function (PDF)

    “Propagation of PDFs”

    In other cases it is not wrong, but inadequate

    Stating the uncertainties associated with the values of measured or computed quantities provides a measure of the reliability of these results.The GUM provides a consistent procedure and is accepted worldwide.The GUM is useful in practice as it provides acceptable solutions in all cases where a linear model function is adequate.In cases where the simple GUM approach is said to fail, one often finds that the GUM simply has not been read in detail, see pages #5, #18 and Annex 1, page #32 (top).GUM versus Error analysis see bottom next page.Incomplete knowledge and PDF(How to express incomplete knowledge? See bottom on page #10.Propagation of PDFs : see “Derivation” of Markov FormulaNote: In the case of transport calculations for dosimetry one is to consider uncertainty contributions not only from the geometrical parameters but also from⇒ cross sections (integral and differential) and⇒ algorithms (especially the estimators used),

    admittedly both is hard to do, but not impossible!However, any expenditure in work is to be judgedby what it returns!

  • 4

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 4

    Evolution of the GUMEvolution of the ‘Guide to the Expression of Uncertainty in Measurement’Walter Bich1, Maurice G Cox2 and Peter M Harris2

    1 Istituto Nazionale di Ricerca Metrologica, 10135 Torino, Italy2 National Physical Laboratory, Teddington TW11 0LW, UK

    Metrologia 43 (2006) S161–S166 (Institute of Physics Publishing (IOP))

    •Measurement model relating functionally one or moreoutput quantities, about which information is required, toinput quantities, about which information is available.

    • Modelling of measurement knowledge about a quantity interms of a probability distribution (PDF).

    • Expectation (estimate) and standard deviation (standarduncertainty) of a quantity characterized by a PD.

    • Use of new information to update an input PDF:Bayes’ theorem.

    • Assignment of a probability density function to a quantityusing the Principle of Maximum (Information) Entropy.

    • Determination of the distribution for an output quantity(or the joint distribution for more than one output quantity)

    using the propagation of distributions.

    Concepts and Principles

    -An introduction to the ‘Guide to the Expression of Uncertainty in Measurement’ and related documents.

    -Concepts and basic principles.-Supplement 1 to the GUM—Propagation of distributions

    using a Monte Carlo method.-Supplement 2 to the GUM —Models with any number

    of output quantities.-Supplement 3 to the GUM’—Modelling.-The role of measurement uncertainty in deciding

    conformance to specified requirements.-Applications of the least-squares method.

    Documents currentlyplanned by JCGM / WG 1

    Joint Committee on Guidance in Metrology / Working Group 1

    For a blown up version of above boxed texts see viewgraph 31.The paper cited above helps a lot to clarify the sometimesambiguous annotations in the GUM-document. The ambiguity isoften caused by the attempt to reconcile the views of classicalstatisticians and modern (rediscovered) Bayesian concepts,which are now fully supported by this article.

    An account of differences between “classical” and Bayesian view can be found in:Gleser L J :Assessing uncertainty in measurement, Stat. Sci. 13 (1998) 277–90and in an more recent and “closer to physics” article byIgnacio Lira and Wolfgang Wöger :Comparison between the conventional and Bayesian approaches to evaluate measurement data, Metrologia 43 (2006) S249–S259.Personal note: To me, only the Bayesian approach supports a consistent treatment and a combination of uncertainties due to stochastic effects (e.g. in Monte Carlo calculation or with repeated measurements), known systematicdeviations and those known from previous measurements or evaluations.

  • 5

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 5

    Evolution of the GUM: Concepts and Principles

    Measurement model relates output quantities, about which information is required, to input quantities, about which information is available.

    ( )rΩ,,EΦ “acts” onPersonObjectDetector

    causes DoseReadingDoseReading

    Cause EffectForward calculation

    Inverse problem

    A measurement model requiresto understand a measurement and itreplaces the inverse problem by a forward calculation!

    The concept of using a measurement model was already stated in the GUM. There, however, with an emphasis on a linear or “linearized” models for one out quantity. One may conjecture, that one reason behind this was, that the central limit theorem (see bottom of page 21) yields only a Gaussian PDF for the output quantity if many input quantities (with comparable relative uncertainties) are combined linearly; i.e. the PDF for the sums or differences of say ten uniform PFDs with the same variance is practically a Gaussian with a ten times as large variance.

    Consider the model Y ≡ fY(X) =X 2, then the full Taylor expansion is given by:2

    2

    2

    )()(

    !21)(

    )(!1

    1)( xX

    xfx

    Xxf

    xfY YYY −∂∂

    +−∂

    ∂+= ξξ

    In the standard framework of the GUM, only the linear term is used:

    E is the symbol forexpectation (see vg.# 7)

    y ≡ fY(x) =x 2 andcX is the first derivative of

    fY(X) with respect to X,evaluated at X=x.

    0since,)(E 22 ==−+= XX cxxcxy ξ

    )(])(E[)( 22222 xucxcyu XX =−= ξ

    See also Note to GUM clause 5.1.2page 20 (bottom) and page 32 (top), please.

  • 6

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 6

    Evolution of the GUM: Concepts and PrinciplesMeasurement model relates output quantities, about which information is required, to input quantities, about which information is available.

    There is no theory on modelling,but there are some helpful general procedures, e.g. K D Sommer and B R L Siebert, Systematic approach to the modelling of

    measurements for uncertainty evaluation, Metrologia 43 (2006), 200–210.B R L Siebert, Uncertainty in radiation protection dosimetry: basic concepts

    and methods, Radiat. Prot. Dosim. 121, (2006), 3-11.

    I believe, the most simple and method is to “play the role of the cause”!

    I am a 252Cf neutron that has just been produced inside the source:How do I reach the surface of the Bonner sphere and then central detector?

    Sommer et al. suggest to visualize the “cause-effect-chain”:

    0 0( )k kG Gδ+INkX OUTkX

    0kG 0kGδ kZδ

    ( )P ∆

    XINX INDδ

    IND

    XIND

    XINSTRδ XM

    + +

    ( )P ∆

    XINX INDδ

    IND

    XIND

    XINSTRδ XM

    ++ ++

    XSRC

    SRC

    X SRC∆ ( )P

    XSRC0+

    -

    for material measures only

    XSRC

    SRC

    X SRC∆ ( )P

    XSRC0++

    -

    for material measures only

    XIN XOUT

    XT1 XTm... XTj... XTδ ( )P

    TRANS

    h(XIN , XT1, ..., XT m)+XIN XOUT

    XT1 XTm... XTj... XTδ ( )P

    TRANS

    h(XIN , XT1, ..., XT m)++

    They show that in mostcases only a source link (SCR) and an indication (IND) link and a fewTRANS-links are needed.

    The SCR-link models the cause, the IND-link the indication (read off yield).The TRANS-links describe all other influences.

    A systematicapplication ofthis concept isdemonstratedin the abovecited RPDarticle.

  • 7

    The PDF gX(ξ) would be represented ideally by the corresponding capital Greek letter, here Ξ, but this is not customary and one uses the same symbol as for the quantity, here X. In a given context it is generally clear what is meant.For simple calculations it is helpful to introduce the use of PDFs in standard form, i.e. EX STD-PDF = 0 and Var X STD-PDF = 1. Furthermore, many frequently occurring PDFs are symmetric, i.e. the expectation E X 2n+1 = 0, n = 0,1,2… vanishes! The rules for calculating expectation are:

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 7

    Evolution of the GUM: Concepts and PrinciplesModelling of measurement knowledge about a quantity in terms of aprobability distribution (PDF).

    Expectation (estimate) and standard deviation (standard uncertainty)of a quantity characterized by a probability distribution.

    ( ) XgX ≡≡ Ξξ ( ) PDFSTD−+= XxuxX ( ) ( )ξξ ′+⇒ − XxX guxg ,PDFSTD

    ( ) 1d =∫∞

    ∞−

    ξξXg

    ( ) xXξg X ==∫∞

    ∞−

    Edξξ

    ( ) ( ) )(VardE 22 xuXXg X ==−∫∞

    ∞−

    ξξξ

    Normalisation

    Expectation

    ( ) ( )

    ( ) 2121

    21212121,

    EECov

    Edd,21

    XXXX

    XXg XX

    +=

    =∫∞

    ∞−

    ξξξξξξ

    )()()( Cov),(

    21

    2121 xuxu

    XXxxr =Variance and uncertainty

    Covariance and correlation coefficient

    For an example of using this formalism see page #32 (top)

    )()()();()(

    1121 XXXXcXccX

    EEEconstantEE±=±

    ==

    22

    21

    2

    2

    )(E

    )(E2)(E])(E[)(Var

    xX

    xXxXxXX

    −=

    +−=

    −=

    0E)( since;])([E

    PDFSTD

    PDFSTD

    ==+

    XxuxXxux

    )(]))([( 22 xuXxux =+ −PDFSTDE

  • 8

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 8

    Evolution of the GUM: Concepts and PrinciplesUse of new information to update an input probability density function:Bayes’ theorem.

    Bayes’ theorem in modern nomenclature: the posterior pdf gX(ξ |D,I ) taking account of new data Dresults from prior pdf gX( ξ |I ) taking account of prior information Ias product of a constant C, the likelihood l(ξ |D,I ) and the prior pdf.

    ( ) ( ) ( ) ξξξξξ d ,d , IgIDlCIDg XX =

    Thomas Bayes, 1702 - 1761, English reverend, found this theorem. It was used by Laplace. Later statisticians did not accept this theorem.

    The theorem was rediscovered by Jeffreys (1938) and is since gaining increasingly acceptance asthe theoretical basis for inference.

    Admittedly, there are some controversial discussions under the heading “Bayesians versus Frequentists”. Scientists should be open minded to all arguments and irrespective of the methods used they should in the end validate their results, or more exactly try sufficiently hard to find contradicting facts. In the end, natural science is the art of inferring.Bayesian methods of inference reflect the process of learning. They are therefore well suited for analysing measurement uncertainty, that is in principle an iterative process in that more and more information is gathered. We define probability according to Bernoulli. A PDF is then not more and not less than simply a mathematical representation of our expectation on that “we would bet”, as we inferred it from all relevant facts known to us. This interpretation leads to two important requirements:

    One is to include all relevant facts known andOne is always to be aware that one’s knowledge is incomplete.

    The qualifier “relevant” means simply that in evaluating uncertainty one aims for good reasons at a finite accuracy of about 5%.

    See bottom part of page #32 for an example on the use of Bayesian methods.

  • 9

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 9

    Evolution of the GUM: Concepts and PrinciplesAssignment of a probability density function to a quantity using thePrinciple of Maximum (Information) Entropy..

    ( ) ( )in

    iin plnpp,...,p,pH ∑

    =

    −=1

    21

    The principle of maximum information entropy yields:a rectangular (uniform) pdf if one only knows:the values ξ of the quantity X are contained in an intervala Gaussian (normal) pdf if one only knowsthe best estimate x and u(x)

    formulated by Shannon ensuresthe most probable PDF that takes only account of the information given

    The PME (here for discrete probabilities):

    A pioneer in using the PMEwas the physicistE. T. Jaynes (1922 - 1998)

    Examples for using Bayes’ theorem and PME are provided in the proceedings of the previous workshop:Intercomparison on the usage of compu-tational Codes in Radiation dosimetry,G. Gualdrini and P. Ferrari (editors)2004 (ENEA), ISBN 88-8286-114-7

    In a certain way, using the PME is equivalent to using Occam’s razor:“entia non sunt multiplicanda praeter necessitatem“

    which, translated to our needs, means, that one shall not make more assumptions as necessary or more coined:Use only the information you are really given, (but use the entire information!).Now, if our only information is that the value of a quantity is bounded, any deviation from a uniform PDF would be equivalent to additional information.

    One can work in existing additional informationin the form of moments that are then used asconstraints in a Lagrange variation procedure;e.g. proceedings previous Bologna workshop, p. 20-21.

    A very well written book, that explains PME andBayesian approaches is: Data Analysis: A Bayesian Tutorialby Devinder S. Sivia , Oxford Science Publications,Oxford University Press, ISBN 0-19 851 889-7

    In addition: google for ” G.D. Agostini ∩ CERN ∩ Bayes”.

  • 10

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 10

    Evolution of the GUM: Concepts and PrinciplesDetermination of the distribution for an output quantity (or the jointdistribution for more than one output quantity) using the propagation ofdistributions.

    ss ss

    α β

    u= su= s

    values o f m easurand

    U = 2s

    probab ility density fo r m easurand

    best estim ate(resu lt o f m easurem ent)

    p rob . fo r the valueto b e betw een α and β

    probability to contain value ofmeasurand can be calculatedfor any chosen interval!

    Coverage intervalwith corresponding probability

    The value of the measurand is after the measurement fixed,but our knowledge about the value is incomplete.The PDF expresses this incomplete knowledge.

    The concept of probability used here is that of J. Bernoulli and best described as the Degree of belief one has in view of the given information.If you do not like the word “believe” replace it by “trust”.

    Jacob Bernoulli (lived in Basel, Switzerland, 1654– 1705) and is best known for the workArs Conjectandi (The Art of Conjecture),

    that contains studies on games and probability.

    The concept of probability as“measurable” ratio of the number of “successes”to thenumber of “possibilities”,is not sufficient, as it would notallow modern risk analysis for“events that never yet occurred”and hopefully never occur!

  • 11

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 11

    Evolution of the GUM: Documents currently planned Documents currently plannedby JCGM / WG 1

    An introduction to the GUM and related documents.Concepts and basic principles.Supplement 1 to the GUM:

    Propagation of distributions using a Monte Carlo method.Supplement 2 to the GUM:

    Models with any number of output quantities.Supplement 3 to the GUM

    Modelling.The role of measurement uncertainty indeciding conformance to specified requirements.Applications of the least-squares method.

    An introduction to GUM and related documents will provide the historical background of the development of the GUM. Concepts and basic principleswill in detail describe theconcepts of the model of measurement (i.e. the functional inter-

    relation of input and output quantities) and of modellingmeasurement knowledge about a quantity in terms of aprobability distribution function and the

    basic principles of assigning PDFs to quantities,using Bayesian theory and the principle of maximum(information) entropy (PME) .

    Supplement 1 to the GUM will be published soon. It describes the use of a Monte Carlo Method to propagate the PDFs for the input quantities to obtain the PDF for the output quantities. The section on Monte Carlo in this presentation is based on this supplement.Supplements 2 and 3 will provide further guidance.The role of … will be addressed in the section on conformity, see pages#26-#28, please.The least squares method is often applied in evaluating key comparisons, see page #26 (bottom), please.

  • 12

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 12

    “Derivation” of Markov Formula

    654321

    1

    2

    3

    2

    5

    6

    654321

    1

    2

    3

    2

    5

    6

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    654321

    7654321

    8765432

    9876543

    10987652

    111098765

    1211109876

    654321

    7654321

    8765432

    9876543

    10987652

    111098765

    1211109876

    Fair dice ⇒ p(i)= 1/6 for i∈[1, 6]PDFs for the “quantities”number shown in a throw , X1 and X2 , are discrete “uniform”.

    X1X2

    How do we get the PDF for Y=X1+X2 ?

    (1) Calculate probabilities of all pairs (i1, i2)and the corresponding sums

    “Quantity” Y : sum of the numbers shown

    (2) Compute the cumulative distribution function

    12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    1 3 6 10 15 21 26 30 33 35 36

    2 times

    1 + 2

    We use simple discrete probability distributions as this enables us to calculate everything easily and exactly.Assume we know that the dice are fair. With other words we know only the boundaries of the possible values. Therefore the PME tells us that the probability distribution function is uniform (rectangular). Since six values are possible and none is preferred against any other values, the probability for any value to be on top after throwing one dice is 1/6. Furthermore assume we know that one can throw two dice in a way that there is no correlation between the results for either dice. This given knowledge is the basis for the upper right table that contains the probabilities for the combinations, here they have the value 1/36 since the number obtained with the first dice is independent of the number obtained with the second dice, i.e. we get simply the product of the probabilities.In the right table at the bottom the result, here the sum of the top values shownby the thrown dice. It shows, that a sum value of 2 occurs once, a sum value of 3 twice and so on.In the left table on the bottomshows the resulting

    Cumulative Probability Function.

    ( ) ( )∑=

    =I

    iiI xPDFxCDF

    1discretediscrete

  • 13

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 13

    “Derivation” of Markov Formula

    ξ1

    ξ2

    η

    ηη12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    1 3 6 10 15 21 26 30 33 35 36

    12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    1 3 6 10 15 21 26 30 33 35 366 10 15 21 26 30 33 35 36

    .d)()( ∫ ∞− ′′=ξ

    ξξ ξgGX XCumulative distribution function (CDF):

    ,d))(()()( ∫∞

    ∞−−= ξξξ fHgGY ηη X

    ⎩⎨⎧ ≥

    =.otherwise,0

    ,0,1)(

    zzH

    )()(dd ηηη YY

    gG =

    ( )∫∞

    ∞−−= ξξ d))((

    dd ξX fHg ηη

    ∫∞

    ∞−−= .d))((δ)( ξξ ξX fg η

    η = ( η1,…, ηn)T

    ξ = ( ξ1,…, ξN)T

    Consider now model Y=X1X2 and Gaussian PDFs for X1 and X2 , Cov(X1X2 )=0.

    This to say that Markov formula holds any number of in- and output quantities

    654321

    1

    2

    3

    2

    5

    6

    654321

    1

    2

    3

    2

    5

    6

    361

    361

    361

    361

    361

    361

    361361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361361361361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    654321

    7654321

    8765432

    9876543

    10987652

    111098765

    1211109876

    654321

    7654321

    8765432

    9876543

    10987652

    111098765

    1211109876

    η=ξ

    1ξ2

    To be continuednext page, bottom.

    2121 )( )( 21 ξξξξ ∆∆XX ggp =Ordinate:

    1/36,everywhere

  • 14

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 14

    GUM Framework follows from Markov Formula.dd))(()δ()( 1 NY ξξfηgg LξξX −= ∫

    ∞−η

    ( ) ( ) .dd)()(d)()(

    ,dd )()( d)(

    1222

    1

    NY

    NY

    ξξyfgygyu

    ξξfggy

    L

    L

    −=−=

    ==

    ∫∫∫∫

    ∞−

    ∞−

    ∞−

    ∞−

    ξξ

    ξξ

    X

    X

    ηηη

    ξηηη

    ( )iiN

    iiN xcyf −+= ∑

    =

    ξξξ1

    1 ),..,(⇒Linear model or linearized via Taylor expansion

    0...dd )( ),...,(... 1N1,...,1 =⇒∀ ∫ ∫∞

    ∞−

    ∞−NiiXX -xgi N ξξξξξ

    yyg NXX N =∫ ∫∞

    ∞−

    ∞−

    ξξξξ ...dd ),...,(... 1N1,..., 1⇒ GUM-solution

    If linear then:

    One can also show that

    ∑ ∑∑−

    =

    +==

    +=1

    1

    1

    11

    22 )Cov(2)Var()(N

    i

    N

    ijjiji

    N

    iii ,XXccXcyu

    Starting point forderivation below

    See next page

    Continued. Identify all elements in the ξ1ξ2 -plane for which η ≤η k (left figure).

    η=ξ

    1ξ2

    2121 )( )( 21 ξξξξ ∆∆XX ggp=Ordinate:

    Use right figure to sum up the probabilities:

    ( ) ( )2121 ξξξξη jii j

    kk pHP ∆∆∑∑ −=

    ( ) kk P=η CDF ( ) 0 and0, 1 →−→ + kkji ηη∆∆ ( ) ( )ηη Yk G→ CDF

    Imagine the left figure in a aquarium that you fill slowly with water, identify all

    submersed elements in the ξ1ξ2 -plane for which η ≤η k (left figure).

  • 15

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 15

    GUM Framework follows from Markov Formula

    .dd))(()δ()( 1 NY ξξfηgg LξξX −= ∫∞

    ∞−η

    ( ) ( ) .dd)()(d)()(

    ,dd )()( d)(

    1222

    1

    NY

    NY

    ξξyfgygyu

    ξξfggy

    L

    L

    −=−=

    ==

    ∫∫∫∫

    ∞−

    ∞−

    ∞−

    ∞−

    ξξ

    ξξ

    X

    X

    ηηη

    ξηηη

    One can also show that a linear model leads to the GUM formula:

    ∑ ∑∑=

    +==

    +=N

    i

    N

    ijjiji

    N

    iii ,XXccXcyu

    1

    1

    11

    22 )Cov(2)Var()(

    If, as in pure physics, we do not to determine a coverage interval, i.e. the expanded uncertainty, then our job would be done after simply computing the integrals:

    i

    iYi X

    xfc∂

    ∂=

    )(

    Insertion of the linear model, using the linear Taylor expansion (bottom page #5)yields:

    use vector notation and introduce the variance-covariancematrix, i.e. the uncertainty matrix, for the input quantities:

    Ux with the elements u(xi,xj) ≡ uij ; uii≡u2(xi).

    The linear(ized) model results then in

    ( ) cUc xT2 =yu

    NNNXX yfgyu N ξξξξξξ ...d d )),...,( (),...,( ...)( 12

    11,...,2

    1∫ ∫∞

    ∞−

    ∞−

    −=

    ( ) NN

    iiiNXX xcgyu N ξξξξξ ...d d ),...,( ...)( 1

    2

    11,...,

    21∫ ∫ ∑

    ∞−

    ∞− =⎟⎟⎠

    ⎞⎜⎜⎝

    ⎛−=

    ( )( )( ) NjiNXXN

    i

    N

    jji xxgccyu N ξξξξξξ ...d d ),...,( ...)( 11,...,

    1 1

    21∫ ∫∑∑

    ∞−

    ∞−= =

    −−=

    ( )( )( ) jijiNXXji xxgxxu N ξξξξξξ ...d d ),...,( ...),( 1,...,1∫ ∫∞

    ∞−

    ∞−

    −−=

    Vector notation:

    ( )T....,1 Nxx=xT

    )(....,)(

    1

    1⎟⎟⎠

    ⎞⎜⎜⎝

    ⎛∂

    ∂∂

    ∂=

    N

    NYY

    Xxf

    Xxfc

    ⎟⎟⎟

    ⎜⎜⎜

    ⎛=

    nNN

    N

    uu

    uu

    ............

    ...

    1

    111

    xU

    and this isequivalent to ∑ ∑∑

    =

    +==

    +=N

    i

    N

    ijjiji

    N

    iii ,XXccXcyu

    1

    1

    11

    22 )Cov(2)Var()(

  • 16

    How well (En,Ωn), i.e. the sourceknown?

    Passage through gasket

    Passage through air

    Chance to hit the Bonner sphere

    surface, Moderation in Bonner sphere

    Chance to reach the central sensor,

    Chance to react inside the sensor,

    and chance that event is registered

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 16

    GUM framework: Summary using an exampleI am a 252Cf neutron that has just been produced inside the source:How do I reach the surface of the Bonner sphere and then central detector?

    I have the property (En,Ωn) andhave a chance to“ yield a detection event”

    I, the neutron, pass through the gaskettravel through airreach the surface of the Bonner sphere,will be moderatedhave a chance to reach the central sensor,then a chance to react inside the sensor,and if I am lucky this event is registeredand I cause “a detection event”

    First consider an idealised case,but note possible influences

    Subdivide the “journey” in sub-trips, i.e. sub-models, butnote possible correlations

    Recommended procedure:

    Compile what you know aboutidentified influencing quantities Xiand state their uncertainties

    Compute the uncertainty

    Analyse and judge the resultand, if needed, iterate.

    It is helpful to visualize the cause action chain as indicated on viewgraph 6. Here a possible listing of influences is given.

    Emittance property ofaccelerator or neutronsource. Information fromproducer, operator and monitors

    Geometry, cross sectionsattenuation, multiple scattering:in and out, variance reductionto reach sphere surface and then the detector? Information from design sheets,ENDF data and “error”-files, Code descriptions of possibletallies. And… and…

    All over detection efficiency.Information from producer andcalibration measurements.

  • 17

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 17

    Expanded Uncertainty ⇔ Coverage IntervalCalibration laboratories are to state the expanded uncertainty

    Expanded uncertainty needs to be known for conformity statements

    The standard GUM approach assumes implicitly that thePDF for the measurand is a Gaussian.

    If this is not the case, one needs to compute the PDF for the measurand.

    If one has many input quantities and if their relative uncertainties areof similar size, then one can assume a Gaussian PDF for the measurand.

    .dd))(()δ()( 1 NY ξξfηgg LξξX −= ∫∞

    ∞−η

    Markov formula

    ( )yuYyY

    ≡≡

    VarE

    GUM: ( ) YyUyYUpg ppy

    yY E,E and ,d maxmin

    max

    min

    −=−== +−∫ ηη

    One should not confuse “Coverage Interval” and “Confidence Interval” .

    The expanded uncertainty Up for coverage probability p can be calculated from the CDF for Y by determining, generally numerically, values ηmin,p and ηmax,p that satisfy .)()( pmin,pmax, pGG =− ηη YY

    The values ηmin,p and ηmax,p are the boundaries of the coverage interval. The expanded uncertainty Up is defined by Up =ηmax,p − ηmin,pIn practice, Up is expressed as the product of a coverage factor kp and u(y), i.e., Up = kp u(y). For a Gaussian, the value of the coverage interval for a coverage probability p=0.95 u(y), is equal to 1.96, rounded 2. For a rectangular PDF one obtains kp = 1.64. If the resulting PDF is not symmetric one specifies: and .

    and , i. e. one searches for the shortest interval.This is reasonable as one wants to localise the result as well as possible.

    ( )pp ,min,max ηη −min

    pp yU ,minη−=− yU pp −=+ ,maxη

  • 18

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 18

    Analytical and Numerical Methods: Use of residue theorem

    ( ) ( ) ( )( ) ξξXY df̂gg δη ξ∫∞

    ∞−

    = ( ) ( )ξfηf −=ξˆ

    ( )( ) ( )in

    ii

    ff

    ,01,0 ˆ

    1)(ˆξ

    ξξδξδ′

    −= ∑=

    11,0

    −=ηξ ( ) 2ˆ −=′ ξξf

    ( )( ) 21)(ˆ −−−= ηηξδξδ f

    Example: Y=1/X

    ( )[ ]

    ⎪⎩

    ⎪⎨

    ⎧∈

    = +−−

    else 0

    , if 32)(

    2

    ηηηηη xugY ( )xux 3

    =mηwhere

    In the above example the PDF for X is rectangular (uniform). For the often discussed Model Y=X 2 and again a rectangular PFD for X, one obtains:

    and via appropriateintegration:

    If in the above example the PDF for X is a Gaussian, one obtains the same value for y but

    (see also page 32,top)

    ( ) ].))(3(,))(3[(,1)(3

    1 22 xuxxuxxu

    gY +−∈= ηηη

    )(544)()(and)( 2222 xuxxuyuxuxy +=+=

    ( )( )

    ( )( )⎟

    ⎟⎠

    ⎞⎜⎜⎝

    −+

    === ∫+

    −xuxxux

    xugyY Y 3

    3log321dE eηηη

    η

    η

    ( )[ ] ( ) ( )( ) ( )2

    2222

    31dEVar y

    xuxygyuyYY Y −−

    =−==−= ∫+

    ηηηη

    η

    )(24)()( 22 xuxxuyu +=

  • 19

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 19

    Analytical and Numerical Methods: Substitution of variables

    Example: Y=X1+X2

    G= Region oftendifficult

    ( ) ( ) ( ) d d),(),( )2

    1( )21( )( 2~ 1

    21121,21, 21

    ζζζζψϕζηδζζζζη ∫

    ∂∂

    −−+=G

    XXY ggg

    ( ) ( )( ) ( ) 2/,

    2/,

    21212

    21211

    ζζζζψξζζζζϕξ

    −=≡+=≡Substitution 121 ζξξ =+

    Not always possible

    ( ) ( )21,/, ζζψϕ ∂∂ = = −1/2⎟⎟⎟⎟⎟

    ⎜⎜⎜⎜⎜

    ∂∂

    ∂∂

    ∂∂

    ∂∂

    21

    21

    ζψ

    ζψ

    ζϕ

    ζϕ

    Both analytical methods work only in special cases!

    If both PDFs are rectangular, the integration is fairly simple:

    -6 -4 -2 0 2 4 6ξ 1

    -6

    -4

    -2

    0

    2

    4

    6

    ξ 2

    -6 -4 -2 0 2 4 6ζ 1

    -6

    -4

    -2

    0

    2

    4

    6

    ζ 2

    B=(-a1,-a2)B=(-a1,-a2)

    A=(-a1, a2)A=(-a1, a2) D=( a1, a2)D=( a1, a2)

    C=( a1,-a2)C=( a1,-a2)

    A'=(-a1+a2,-a1-a2)

    B'=(-a1-a2,-a1+a2)B'=(-a1-a2,-a1+a2) C'=(a1-a2,a1+a2)

    D'=D'=(a1+a2,a1-a2)

    II IIII IIIIII I'I' II'II' III'III'

    The result is the well known trapezoidal PDF:

    [ ][ ][ ]

    ,4/)(

    /4,

    ,4/)(

    )(

    2121212

    2

    21211

    2

    2121212

    2

    ⎪⎩

    ⎪⎨

    +∈−+

    −+∈

    +∈++

    =−

    aa, a-aaaa

    aa,a-aa

    a-a,a--aaaa

    gYηη

    η

    ηη

    η

    For details see: Bernd R.L. Siebert:Berechnung der Messunsicherheitmit der Monte-Carlo-Methode,PTB Mitteilungen 111(4), (2001),S.323-337.

  • 20

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 20

    Analytical and Numerical MethodsHigher order Taylor-expansion:

    ( ) ( )

    22 3

    M M M2

    1 1

    2 2

    1...2

    N N

    i j i j i i j

    i j

    f f fx x x x x

    u x u x

    = =

    ⎧ ⎫⎛ ⎞∂ ∂ ∂⎪ ⎪+ + ⋅⎜ ⎟⎨ ⎬∂ ∂ ∂ ∂ ∂ ⎪⎪ ⎭⎝ ⎠⎩

    ⋅ ⋅

    ∑ ∑ In general quite complicated.

    Brute force numerical integration:

    Requires input PDFs in very high resolution.

    Trapezoidal and other integration schemes or use ofFourier transformation is sometimes helpful,but not for many input quantities and non-linear models

    An example for higher order Taylor expansion has been given on page 5 (bottom).

    DO I1 = 1,Mdummy=XSI (I1)DO I2 = 1,M

    dummy=dummy+ XSI (I2)K=(dumy−offset)/delta_aeta+1AETA(K)=AETA(K)+1/M**2

    END DOEND DO

    The most simple “Brute Force” approach would be implemented in FORTRAN by nested Do Loops.

    Consider the model treated on the previous page Y=X1+X2 and assume again rectangular PDFs for X1 and X2 , for ease in demonstration x1=x2 =0 and u(x1)=u(x2). Represent the PDFs by

    { } ( )M

    xurxuX rM)(32∆and∆2

    1)(3,,..., 1 =−+−== ξξξξξ

    As an exercise, consider the Model Y=X1X2 and find out how big M must bein order to obtain a reasonable representation of the PDF for Y.

    In this case take:delta_aeta = ∆ξoffset = The result is a triangular PDF, represented as histogram.

    32−

    21 IIp ξξ ∧for

  • 21

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 21

    A Monte Carlo MethodThe use of a Monte Carlo method for evaluating uncertaintyand expanded uncertainty Maurice G Cox and Bernd R L SiebertMetrologia 43 (2006) S178–S188

    654321

    1

    2

    3

    2

    5

    6

    654321

    1

    2

    3

    2

    5

    6

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    361

    654321

    7654321

    8765432

    9876543

    10987652

    111098765

    1211109876

    654321

    7654321

    8765432

    9876543

    10987652

    111098765

    1211109876

    12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    12111098765432η

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    ⎯36

    CDF

    1 3 6 10 15 21 26 30 33 35 36

    On pages 12 and 13 we useddice to compute PDF and CDFfor the sum of values shownby two dice.

    If we consider insteadrectangular PDFs,imagine dice with aninfinite number of faces,we cannot enumerateall possible combinations.

    In principle, a MCM does nothing else but to infer from a finite number ofall possible combinations the properties of all possible combinations.

    A Monte Carlo method (MCM) computes a frequency distribution that is, in finite resolution, an estimate of the PDF one wants to know.

    The PDF Y is sampled M times, this yields: { }MrY ηηη ,...,,..,~ 1=

    ∑=

    −=M

    rrMY

    1

    1~E η is not equal to EY, it is a random variable.Y~E

    The assumption is made that “converges” to EY as M increases.Y~E

    For reasons seen on the bottom of the next page, we consider now H such random variables:

    ∑=

    −=M

    r

    hr

    h MY1

    )(1)(E η and ask how fast does ∑=

    −=H

    h

    hH YHy1

    )(1)( E

    converge to EY as H increases?

    The answer will be given by the central limit theorem, bottom page 22

    Any estimator on yields a value that depends on M and the random numbers used to generate the M samples of Y. For instance, consider the arithmetical mean value of the samples as estimate of the expectation of Y :

    Y~

    The notation (h) is used in the GUM supplement, it will not be used below.

  • 22

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 22

    A Monte Carlo Method

    ξ 1

    ξ 2

    ξ 3

    η

    Y = f ( , , )X 1 X 2 X 3

    Need a procedure that in the statistical mean hasthe same property as fair dice, i.e. that yieldsrepresentative samples for all input quantities

    ),...( 1 Nr f ξξη =

    continued on bottom of page 23

    * Taken from Karl Bury: Statistical Distributions in Engineering, Cambridge University Press(1999) ,ISBN 0 521 63506 3. Adapted to nomenclature in GUM Supplement I, however ,

    using index h instead of (h).

    The mathematical basis of a Monte Carlo Method is the

    Central Limit Theorem*

    If H independent random variables Yh are given with E(Yh ) = yh ,finite variances and finite third moments

    then is asymptotically Normally distributed with

    )()(Var 2 hhh yuY =

    ∑=

    =H

    hhhYaY

    1

    ~

    ( ) ( ) ∑∑==

    ====H

    hhh

    H

    hhh yuayuYyayY

    1

    222

    1)()~(~~~ Varand E

    * Taken from Karl Bury: Statistical Distributions in Engineering, Cambridge University Press(1999) ,ISBN 0 521 63506 3. Adapted to nomenclature in GUM Supplement I, however ,

    using index h instead of (h).

    The mathematical basis of a Monte Carlo Method is the

    Central Limit Theorem*

    If H independent random variables Yh are given with E(Yh ) = yh ,finite variances and finite third moments

    then is asymptotically Normally distributed with

    )()(Var 2 hhh yuY =

    ∑=

    =H

    hhhYaY

    1

    ~

    ( ) ( ) ∑∑==

    ====H

    hhh

    H

    hhh yuayuYyayY

    1

    222

    1)()~(~~~ Varand E

    Below, we use generally ah=1/H

  • 23

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 23

    A Monte Carlo Method

    Statistical weight w=1:Generate a pseudo random number ρ∈[0,1] ( )ρξ 1−= XG

    ( )( ) ( ) ρρξρξ dd gg =

    g(ρ)=1 for ρ ∈ [0,1]and 0 else

    ( ) ( ) ρξζζξ

    ==∫∞−

    Gg d

    ( )ρξ 1−= G-2 -1 0 1 2

    ξ

    0

    0.2

    0.4

    g D (ξ

    )

    0.5

    1

    GD (

    ξ )

    For most PDF one finds the inverse functions in libraries. Most famous is Box-Muller-algorithm for producing normal, i.e. Gaussian, samples.

    Central limit theorem -continued-The central limit theorem is the basis for both, the standard GUMapproach and Monte Carlo Methods.Given many input quantities with finite variances and third moments,the PDF for the output quantity tends to be a Gaussian.If the input quantities have finite variances and third moments one canconsider any sample* as a random variable with anexpectation EY=y and a variance u2(y) that is independent from any nr’ if .r’≠ r; * this independence may be not given, if other than analogue sampling techniques are employed.

    ( )rNrr f ,,1 ,...,ξξη =

    If these conditions are met then (see also page 21 bottom) we arrive at:

    andyM

    yM

    rr

    M

    M⇒= ∑

    =∞→ 1

    1 η)(lim ( ) ( )yuM

    yu MM

    22

    2 1⇒∞→

    )(lim

    ( ) ( )yuyM

    M

    r

    MrM

    2

    1

    2

    11

    ⇒−− ∑=∞→

    )(lim ηu (y) ≡ standard deviation (STDD) of Yu(y(M)) ≡ experimental STDD of the mean.

  • 24

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 24

    A Monte Carlo Method

    Sufficientnumber ofsamples

    ⇓Frequencydistribution

    becomesgood

    approximationto

    probabilitydistribution

    ☺∞→→ ∫

    +

    Sgkg~k

    k

    YY for d )()( 1

    ηηη

    η

    In practice,sufficient convergenceis obtained for S

  • 25

    0 20 40 60 80 100i sorted

    3.6

    3.9

    4.2

    4.5

    4.8

    5.1

    5.4

    U

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 25

    A Monte Carlo Method

    • Input of parameter;

    • Computation of sensitivity coefficients;

    • Get samples from the PDFs for input quantities

    (preferably using G-1Xi(ρ));

    • Get η as a function of ξ1, ξ2,...,ξN (linearized);

    • Get η as a function of ξ1, ξ2,...,ξN (model);• Sorting (check convergence);

    • Statistical analysis;

    • Output of results.

    “Flow chart”

    DO LOOP

    Notnecessary,but highly recommended

    The coverage interval can be found be sorting a buffer of samples. The figure below is based o a buffer size of 2000. Brown dash curves show min/max values, symbols selected examples and yellow band is result after 100 batches.

    ( ) ( ){ }[ ]100,1

    1900min

    −+=

    sorted

    sortedsorted0.95 i

    iiU ηη

    Yellow band:upper line U+u(U)lower line U−u(U)

    min/max is based on 1600 Batches

    max

    min

  • 26

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 26

    Conformity

    Calibration hierarchy: from thenational standard to the product

    NMIassures

    traceabilityto SI units

    One can only meaningfully compare

    if all participants statetheir uncertainties consistently!

    organises key comparisons

    among NMIs

    organiseskey comparisonamong NMIs

    SI units

    Product reliability and safetyrequiresConformity Testingof Measuring Instruments

    product

    However, Monte Carlo methods are also increasingly used in the evaluationof key comparisons. One reason is, that above relations require that thePDF for all quantities Yi (measured by the participants) are Gaussians.Another reason is that sometimes the median of the participants’ values istaken as best estimate. In such cases, that from the metrological point ofview, should be avoided, use a Monte Carlo method to compute the PDFfor Z . Generate a sample of possible participants’ values,

    The method suggested by JCGM for evaluating key comparisons is weighted least squares, using the uncertainties associated with the participants results to construct the weights:.

    ∑ ⋅=N

    iii ywz

    The best estimate for the value of the quantity measured by N participants is:

    where( )

    ∑ −−

    = N

    ii

    ii

    yu

    yuw)(2

    2

    and ( ) ( ) ∑=N

    iii yuwzu

    222

    rNr ,,1 ,...,ηη and sort it in ascending order. Take

    ( )

    ( ) even. 1 , odd 1

    sortsortsortsort

    sort

    NNiNi

    NNi

    iir

    sortir

    +==+=

    +==

    +−+− 2

    121,

    21

    21, **

    ηηζ

    ηζ

  • 27

    Consider a sphygmomanometer as example. The manufactures controls theproduction process, but nevertheless if one and the same “exactly known”blood pressure were measured with different specimens of that instrument,one would find a spread of values, often well describable by a lognormal PDF.The acceptable tolerance in a measurement on patients be ±8 mmHg.If such an instrument is calibrated, an uncertainty will be associated with anymeasurement, too. The resulting expanded uncertainty should be significantlysmaller than the tolerances. In the Figure above we discuss a simplified situation. The PDF for the specification is a Gaussian and the PDF for the quantity measured in calibration is of trapezoidal shape. If the specification of the instrument selectedfor calibration is well within the tolerances,then it would appear to be conform with therequirement, i.e. it complies, even if the un-certainty associated with the value measuredin calibration procedure is large.But if the this specification is near the tolerancelimits, false acceptance or rejection may happen.

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 27

    Conformity Product reliability and safetyrequires Conformity Testing

    product specification ξP

    GL GU

    Up

    RL RU

    Up

    RL RU

    Possible calibration values: ξmPossible product values: ξP

    Y=Xm+Xp

    Up

    RL RU

    Up

    RL RU

    Up

    RL RU

    Up

    RL RU

  • 28

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 28

    Conformity

    IF ξm < TL THEN i = 3IF TL ≤ ξm THEN i = 2IF ξm > TU THEN i = 1IF η > GL THEN j = 1IF GL ≤ η THEN j = 2IF η > GU THEN j = 3

    i

    j

    TL TU

    TL TU

    GU

    GL

    GU

    GL

    RP,U

    RP,L

    Correct rejection

    Correctacceptance

    RU,URU,L

    Source:

    Zinner ( )PPg ξ

    ( )CCg ξ

    TU

    TLGL

    GU

    PDF convolution

    Correctacceptance

    R U,URP,L

    R U,LRP,U

    Correct rejection

    Correctrejection

    ( )CCg ξ

    ( )PPg ξ

    The risk that an instrument is falsely rejected is called producer risk. The risk that an instrument is falsely accepted is called user risk. These risks can be computed: Legislation or agreement serve to limit both risks adequately. Analytical computations lead to in general difficult convolution problems:

    Adapted from: Sommer ,K-D. et al.:Messunsicherheit und Konformitätsbe-wertungen im gesetzlichen Messwesen. PTB Mitteilungen 116 (1) 2006, p 40-49

    However, using Monte Carlo Methods, the convolution is simple:

    ( ) ( ) ( )PPCCY ggg ξξη ⊗=

    However, using Monte Carlo Methods, the convolution is simple:Assume a product specification and a calibration result

    and form a possible value of the resulting calibration measurement on the selected instrumentSorting as shown in the above box (IF… ) yields the contributions:

    ( )rPPrP G ,1, ρξ −=( )rCCrC G ,1, ρξ −=.,, rCrCr ξξη +=

    3

    2

    1

    1 2 3

    For (i, j) = (1, 3), (1, 1) (3, 1) or (3, 3) add to correct rejection,For (i, j) = (2, 3) add to correct acceptation,For (i, j) = (1, 2) or (3, 2) add to user risk andFor (i, j) = (2, 3), (2, 1) add to producer risk.

  • 29

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 29

    Summary

    Evolution of the GUM (JCGM / WG 1)

    “Derivation” of Markov Formula

    GUM Framework follows from Markov Formula

    Expanded Uncertainty ⇔ Coverage Interval

    Analytical and Numerical Methods

    A Monte Carlo Method

    Conformity

    .dd))(()δ()( 1 NY ξξfηgg LξξX −= ∫∞

    ∞−η

    See viewgraph 31

    Measurements serve to gain information about a measurand. In general, this information cannot provide exact values of the measurand. Therefore, before one gives this information in a responsible manner to others or uses it for oneself one ought to quantify the incompleteness of that information. For that purpose one associates a measurement uncertainty with the result of a measurement. Mutatis mutandis, this applies to computations, too.Although in general only an approximation, the GUM procedure for thepropagation of uncertainties is sufficient in praxis if certain conditions are met.However, the general „determination of measurement uncertainty“ requires the propagation of the probability distribution functions for the measured input quantities.

    A Monte Carlo Method is the ideal tool for this task.The JCGM considers now, as documented in various papers by its members and especially in the soon to be published first supplement, that uncertaintyevaluation is based on the concept to assign PDFs for quantities using the PME and Bayes’ theorem. These PDFs reflect logically consistent the principally incomplete knowledge about the values of the quantities. The propagation of these PDFs according to the measurement model for output quantities provides the PDFs for these output quantities.Uncertainty and coverage interval can be computed from such PDFs and be used for stating the compliance within set limits.

  • 30

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 30

    ConclusionI am a Monte Carlo method,who are you?

    You should know me,I am your result!

    A Monte Carlo Methodis the method of choicefor “propagating PDFs”of input quantities tocompute the PDF for theoutput quantities.

    Thank you for your [email protected]

    [email protected]

    The concept, to express the incomplete knowledge abut thevalue of a measurand by a PDFand to relay on Bayes’ theoremand the PME is theoreticallysound and in practice powerful.

    Monte Carlo Methods are in principle simple.However, convergence should be checked carefullyand

    the stochastic uncertainties associated with the computedestimates of the expectancethe variance andthe width and location of the coverage interval

    of the PDF for the output quantity should be stated carefullyand it should be documented how many samples were used,how large the buffer for computing the width and location of thecoverage interval was.

  • 31

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 31

    Evolution of the GUM: Concepts and Principles

    •Measurement model relating functionally one or moreoutput quantities, about which information is required, toinput quantities, about which information is available.

    • Modelling of measurement knowledge about a quantity interms of a probability distribution (PDF).

    • Expectation (estimate) and standard deviation (standarduncertainty) of a quantity characterized by a PDF.

    • Use of new information to update an input PDF:Bayes’ theorem.

    • Assignment of a probability density function to a quantityusing the Principle of Maximum (Information) Entropy.

    • Determination of the distribution for an output quantity(or the joint distribution for more than one output quantity) using the propagation of distributions.

    Taken from:Evolution of the ‘Guide to the Expression of Uncertainty in Measurement’Walter Bich1, Maurice G Cox2,3 and Peter M Harris21 Istituto Nazionale di Ricerca Metrologica, 10135 Torino, Italy2 National Physical Laboratory, Teddington TW11 0LW, UK3 Author to whom any correspondence should be addressed.Metrologia 43 (2006) S161–S166. (Publisher: Institute of Physics)

    Evolution of the GUM: Documents currently planned by JCGM/WG 1

    -An introduction to the ‘Guide to the Expression of Uncertainty in Measurement’ and related documents.

    -Concepts and basic principles.-Supplement 1 to the GUM—Propagation of distributionsusing a Monte Carlo method.

    -Supplement 2 to the GUM —Models with any numberof output quantities.

    -Supplement 3 to the GUM’—Modelling.-The role of measurement uncertainty in deciding

    conformance to specified requirements.-Applications of the least-squares method.

  • 32

    Bernd R.L. GUM-based Uncertainty Evaluation in Radiation Dosimetry: Siebert Survey on Methods and Practices 32

    Annex 1 And again the model Y=X 2Here the application of

    ( ) XgX ≡≡ Ξξ ( ) PDFSTD−+= XxuxX ( ) ( )ξξ ′+⇒ − XxX guxg ,PDFSTDand the calculus with E (expectation) and Var (Variance) is demonstrated.

    ( ) ( )( )2STD)(EE XxuxY += ( ) ( )2STD2STD2 )()(2EE XxuXxuxxY ++=( ) ( ) 1E and 0E 2STDSTD == XX ( ) )(E 22 xuxY +=

    ( ) ( )( ) ( )( ) 2STD4STD )(E)(EVar XxuxXxuxY +−+=( ) 0E about symmetric If 3STD =⇒ XxX

    ( ) ( )( )[ ]1E)(4)(Var 4STD422 −+= XxuxxuY

    ( )( ) 8.1E

    3E4

    rRectangula

    4Gauß

    =

    =

    X

    X

    x Supplement 2 An application of Bayes’ theorem

    Calculation of uncertainty in the presence of prior knowledgeClemens Elster, Physikalisch-Technische Bundesanstalt BerlinMetrologia 44 (20076) S111–S116.

    ( )( )

    ∫∫∫

    ∫∫∫ ∫∫ ∫

    ∫ ∫ ∫∫∫

    −=−=

    −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−

    =

    −=

    −==

    ξξξ

    ξξξξ

    ξξξ

    ξξξξ

    ξξξ

    ξξξ

    ξξξ

    ξξξ

    X

    X

    X

    X

    X

    X

    X

    X

    dgfg

    dgfgfydgyyu

    dgfg

    dgfgf

    ddfgg

    ddfgg

    dddfgg

    dfggdgy

    Y

    YY

    Y

    Y

    Y

    Y

    Y

    YY

    )())((

    )())(())(()()()(

    )())((

    )())(()(

    ~))(~()()~(

    ))(()()(

    ~))(~()()~(

    ))(()()()(

    0

    0222

    0

    0

    0

    0

    0

    0

    ηηη

    ηηδη

    ηηδηη

    ηηηδη

    ηδηηηηη

    If one has prior knowledge about a measurand, given by the PDF ,one can use Bayes’ theorem; this leads after some calculations to:

    )(0 ηYg

    For

    details

    please

    consult

    the

    full

    paper.