G4ConvergenceTester and ExN03Con KOI, Tatsumi SLAC National Accelerator Laboratory 1Geant4...
-
Upload
bruno-barrett -
Category
Documents
-
view
217 -
download
0
Transcript of G4ConvergenceTester and ExN03Con KOI, Tatsumi SLAC National Accelerator Laboratory 1Geant4...
Geant4 Collaboration Workshop 2010-10-6 1
G4ConvergenceTester and
ExN03Con
KOI, TatsumiSLAC National Accelerator Laboratory
Geant4 Collaboration Workshop 2010-10-6 2
"G4ConvergenceTester“and “ExN03Con”
• G4ConvergenceTester provides several information assisting user understanding of convergence level of his/her result, like MCNPs.
• “ExN03Con” is an example of the usage of G4ConvergenceTester.– examples/extended/analysis/N03Con/
• They have been included in Geant4 since v9.0 (2007-Jun)
Geant4 Collaboration Workshop 2010-10-6 3
How to Use#include "G4ConvergenceTester.hh“void ExN03RunAction::BeginOfRunAction(const G4Run* aRun){,,,,,, Eabs_tally = new G4ConvergenceTester(); Egap_tally = new G4ConvergenceTester(); Labs_tally = new G4ConvergenceTester(); Lgap_tally = new G4ConvergenceTester();}void ExN03RunAction::fillPerEvent(G4double EAbs, G4double EGap, G4double LAbs, G4double LGap){,,,,, Eabs_tally->AddScore( EAbs ); Egap_tally->AddScore( EGap ); Labs_tally->AddScore( LAbs ); Lgap_tally->AddScore( LGap );}void ExN03RunAction::EndOfRunAction(const
G4Run* aRun){,,,,, Eabs_tally->ShowResult(); Eabs_tally->ShowHistory(); Egap_tally->ShowResult(); Egap_tally->ShowHistory();
Labs_tally->ShowResult(); Labs_tally->ShowHistory(); Lgap_tally->ShowResult(); Lgap_tally->ShowHistory();}
Geant4 Collaboration Workshop 2010-10-6 4
Output ShowResult()values and logical tests
EFFICIENCY = 0.996338MEAN = 368.83VAR = 3627.76SD = 60.2309R = 0.0025516SHIFT = -86.9154VOV = 0.00310814FOM = 2201.43THE LARGEST SCORE = 431.408 and it happend at 1425th eventAffected Mean = 368.846 and its ratio to orignal is 1.00004Affected VAR = 3627.83 and its ratio to orignal is 1.00002Affected R = 0.00255121 and its ratio to orignal is 0.999846Affected SHIFT = -86.9072 and its ratio to orignal is 0.999905Affected FOM = 2201.43 and its ratio to orignal is 1
MEAN distribution is not RANDOMr follows 1/std::sqrt(N)r is monotonically decrease 1r is less than 0.1. r = 0.0025516VOV follows 1/std::sqrt(N)VOV is monotonically decrease 1FOM distribution is not RANDOMSLOPE is large enoughThis result passes 6 / 8 Convergence Test.
Geant4 Collaboration Workshop 2010-10-6 5
Output ShowHistory()trend
i/16 till_ith mean var sd r vov fom shift e r2eff r2int1 255 368.253 3351.08 57.8885 0.00982485 0.0426828 2381.54 -75.4414 1 0 9.61507e-052 511 366.976 3686.51 60.7166 0.00731197 0.0196636 2172.34 -76.4552 0.998047 3.82216e-06 4.95383e-053 767 366.078 3841.16 61.9771 0.00610909 0.0144748 2075.49 -82.2678 0.996094 5.10621e-06 3.21662e-054 1023 366.095 3715.83 60.9576 0.00520336 0.0103351 2133.71 -78.5195 0.99707 2.86943e-06 2.41791e-055 1279 367.283 3592.01 59.9334 0.00456103 0.00905412 2216.23 -80.3134 0.996875 2.44906e-06 1.83376e-056 1535 365.548 3973.11 63.0326 0.00439972 0.0073182 1990.73 -84.6719 0.996094 2.5531e-06 1.67918e-057 1791 365.964 4016.7 63.3774 0.00409098 0.00677035 1969.38 -89.0621 0.99442 3.13151e-06 1.35952e-058 2047 366.099 3942.82 62.7919 0.00379001 0.00596075 2005.12 -88.1626 0.994629 2.63677e-06 1.17204e-059 2303 366.693 3854.07 62.0812 0.00352709 0.00543865 2053.75 -88.5994 0.995226 2.08212e-06 1.03528e-0510 2559 367.151 3818.23 61.7918 0.00332634 0.00491706 2075.29 -88.5895 0.995313 1.83968e-06 9.22051e-0611 2815 367.211 3770.15 61.4015 0.00315099 0.00441452 2105.31 -87.405 0.995739 1.51974e-06 8.40546e-0612 3071 367.092 3835.82 61.934 0.003044 0.00398929 2069.86 -87.876 0.995768 1.38338e-06 7.87952e-0613 3327 367.861 3731.13 61.083 0.00287836 0.00377558 2133.28 -87.6999 0.996094 1.17836e-06 7.10409e-0614 3583 368.469 3692.83 60.7687 0.00275483 0.00360362 2160.84 -88.4698 0.995815 1.17267e-06 6.4143e-0615 3839 368.856 3610.19 60.0849 0.00262871 0.0033742 2213.45 -87.4147 0.996094 1.02124e-06 5.88707e-0616 4095 368.83 3627.76 60.2309 0.0025516 0.00310814 2202.06 -86.9154 0.996338 8.97356e-07 5.61173e-06
Geant4 Collaboration Workshop 2010-10-6 6
• Following slides are my presentation at 2005 Geant4 Collaboration workshop
A general assistant tool for the checking results
from Monte Carlo simulations
A general assistant tool for the checking results
from Monte Carlo simulationsKoi, TatsumiSLAC/SCCS
7Geant4 Collaboration Workshop 2010-10-6
Contents• Motivation• Precision and Accuracy• Central Limit Theorem• Testing Method• Current Status of Development• Summary
8Geant4 Collaboration Workshop 2010-10-6
Motivation• After a Monte Carlo simulation, we get an
answer. However how to estimate quality of the answer.
What we must remember is• Large number of history does not valid result
of simulation.• Small Relative Error does not valid result of
simulation
9Geant4 Collaboration Workshop 2010-10-6
Motivation (Cont.)• To provide “statistical information
to assist establishing valid confidence intervals for Monte Carlo results” for users, something like MCNPs did.
10Geant4 Collaboration Workshop 2010-10-6
Subject of this study• Precision of the Monte Carlo
simulation• Accuracy of the result is NOT a
subject of this study
At first we have to define Precision and Accuracy of
simulations
11Geant4 Collaboration Workshop 2010-10-6
True Value Mote Carlo Results
AccuracyPrecision
Precision and Accuracy• Precision: Uncertainty caused by statistical
fluctuation• Accuracy: Difference between expected value
and true physical quantity.
12Geant4 Collaboration Workshop 2010-10-6
Subject of this study (Cont.)
• Precision of the Monte Carlo simulation is subject of this study.
• To state accuracy of simulations, we should consider details of simulation, i.e., uncertainties of physical data, modeling of physical processes, variance reduction techniques and so on.
• To make a generalized tool, we have to limit subjects only for precision.
Accuracy is a subject for most of presentations in this workshop. 13Geant4 Collaboration
Workshop 2010-10-6
Principal of this study is
Central Limit Theorem
14Geant4 Collaboration Workshop 2010-10-6
Central Limit Theorem• Every data which are influenced
by many small and unrelated random effects has normally distribution.
• The estimated mean will appear to be sampled from normal distribution with a KNOWN standard deviation when N approaches infinity.
N
15Geant4 Collaboration Workshop 2010-10-6
Central Limit Theorem (Cont.)
• Therefore, We have to check that N have approached infinity in the sense of the CLT, or not.
• This corresponds to the checking the complete sampling of interested phase space has occurred, or not.
16Geant4 Collaboration Workshop 2010-10-6
This is not a simple static test
butcheck of results from nature of Monte Carlo
simulations
17Geant4 Collaboration Workshop 2010-10-6
Checking Values• Mean
• Variance and Standard Deviation
• Relative error
• Variance of Variance
11
2
2
N
xxS
N
ii
N
iixN
x1
1
N
SSwherex
SR xx
22
2
,
4
2
x
x
S
SSVOV
18Geant4 Collaboration Workshop 2010-10-6
Checking Values (Cont.)
• Figure of Merit
• Scoring EfficiencyRintrinsic and Refficiency
• Shift
• SLOPEFit to the Largest history scores
TRFOM
2
1
NSxxSHIFT i23
2
N
historiesZERONONofnumberq
19Geant4 Collaboration Workshop 2010-10-6
What we check?• Behavior of MEAN• Values of R• Time profile of R• Values of VOV• Time profile of VOV• Time profile of FOM• Behavior of FOM• Value of SLOPE• Value of SHIFT
• Effect of the largest history score occurs on the next history.– MEAN– R (Rintrinsic and Refficiency)
– VOV– FOM– SHIFT
Boolean Answer
Numeric Answer 20Geant4 Collaboration
Workshop 2010-10-6
Of cause, Boolean check is carried out
mathematically (statistically)
valuebehavior
time profile
21Geant4 Collaboration Workshop 2010-10-6
For behaviors and time profiles check
• Derive Pearson’s r from data (results and theoretical values)– r=1(-1), perfect positive (negative)
correlation– r=0, uncorrelated
• null hypothesis is set to uncorrelated• Then, follows student t
distribution of degree of freedom • Checking significance of r with null
hypothesis.• Rejection level of null hypothesis is 68.28%
(1σ)
212 rNrt 2N
22Geant4 Collaboration Workshop 2010-10-6
Example• Checking value: Observable Energy of Sampling
Calorimeter.• Material
– Pb (Lead)-Scinitillator• Thickens
– Pb: 8.0 mm/layer, Sci: 2.0 mm/layer• Layers
– 120 layers– 1 m x 1 m – interaction surface
• Beam– Electon 4 GeV
• Range Cuts– 1 mm
Pb
8mm
2mm
Sci.
・・・・・・・・e-
23Geant4 Collaboration Workshop 2010-10-6
SD
9.2
9.3
9.4
9.5
9.6
9.7
9.8
9.9
10
0 20 40 60 80 100 120
SD
Example 100 histories
mean
77
77.5
78
78.5
79
79.5
80
0 20 40 60 80 100 120
mean
R
00.0020.0040.0060.0080.01
0.0120.0140.0160.0180.02
0 20 40 60 80 100 120
R
VOV
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
0 20 40 60 80 100 120
vov
SD
VOV
MEAN
R
Does not pass most of Boolean tests
24Geant4 Collaboration Workshop 2010-10-6
SD
99.29.49.69.810
10.210.410.610.8
11
0 200 400 600 800 1000 1200
SD
vov
00.00050.001
0.00150.002
0.00250.003
0.00350.004
0.00450.005
0 200 400 600 800 1000 1200
vov
R
0
0.001
0.002
0.003
0.004
0.005
0.006
0.007
0 200 400 600 800 1000 1200
R
mean
77
77.5
78
78.5
79
79.5
80
0 200 400 600 800 1000 1200
mean
Example 1,000 histories
SD
VOV
MEAN
R
Does not pass some of Boolean tests
25Geant4 Collaboration Workshop 2010-10-6
vov
00.000050.0001
0.000150.0002
0.000250.0003
0.000350.0004
0.000450.0005
0 2000 4000 6000 8000 10000 12000
vov
R
00.00020.00040.00060.00080.001
0.00120.00140.00160.00180.002
0 2000 4000 6000 8000 10000 12000
R
SD
99.29.49.69.810
10.210.410.610.8
11
0 2000 4000 6000 8000 10000 12000
SD
mean
77
77.5
78
78.5
79
79.5
80
0 2000 4000 6000 8000 10000 12000
mean
Example 10,000 histories
SD
VOV
MEAN
R
Does not pass one of Boolean tests (SLOPE check)
26Geant4 Collaboration Workshop 2010-10-6
How to apply Energy Spectrum estimation
etc.• Checking each
confidence level of P1, P2, P3, P4,,,,
• Of course, scoring efficiency becomes low.
P1
P2
P3
P4
E
V/E
27Geant4 Collaboration Workshop 2010-10-6
Unfortunately, this tool does not work well with
some deterministic variance reduction
techniques.This is come from
limitation of CLT (means some variance are
required for distribution), so that we can not over
come.
28Geant4 Collaboration Workshop 2010-10-6
And some simulations becomes deterministic
without awaking of users.Please check your
simulation carefully.
29Geant4 Collaboration Workshop 2010-10-6
Current Status of Development
• Most part of developments has been done.
• Following items are remained under development.– Output of testing result– Class or function for minimization of
multi dimensional functions
30Geant4 Collaboration Workshop 2010-10-6
We want to include this tool in Geant4
butwhat category is suite for this
tool?
Run, SD, Hits and its collections,
Tally??
31Geant4 Collaboration Workshop 2010-10-6
Summary• We have successfully developed a
general assistant tool for the checking the results from Monte Carlo simulations like MCNPs.
• Through this tool, users easily know the confidence intervals for Monte Carlo results.
32Geant4 Collaboration Workshop 2010-10-6