1

Lesson 2: RNGs and Choosing from distributionsLesson 2: RNGs and Choosing from distributions

• Random number generators (RNG)Random number generators (RNG)• General need for random numbersGeneral need for random numbers• Pseudo-random vs. randomPseudo-random vs. random• Linear congruential generatorsLinear congruential generators• Extension of period of generator by combining several generatorsExtension of period of generator by combining several generators

• Choosing from distributionsChoosing from distributions• DiscreteDiscrete• Continuous: DirectContinuous: Direct• Continuous: RejectionContinuous: Rejection• Probability mixingProbability mixing• Multidimensional distributionsMultidimensional distributions

• Coding event problems: FORTRAN code MCBASE.FORCoding event problems: FORTRAN code MCBASE.FOR

2

Basic view of MC processBasic view of MC process

• We begin with the same black box drawing:We begin with the same black box drawing:

• Lesson 1 was concerned with the outputs.Lesson 1 was concerned with the outputs.• Lesson 2 is concerned with the inputs.Lesson 2 is concerned with the inputs.

• The quality of outputs depends on quality of inputs.The quality of outputs depends on quality of inputs.

• The rest of the course is about the black boxThe rest of the course is about the black box

3

The quest for randomnessThe quest for randomness• In the early days (1950s) of computer-based Monte Carlo, there In the early days (1950s) of computer-based Monte Carlo, there

was work done on MECHANICAL or ELECTRONIC random was work done on MECHANICAL or ELECTRONIC random number generators to give us “true” binary randomness (0 or number generators to give us “true” binary randomness (0 or 1)=digits of fraction1)=digits of fraction

• Relay flopsRelay flops• Decay rates (odd or even) in Decay rates (odd or even) in • Etc.Etc.

• Many of them “filtered” the answer to correct for non-50/50 Many of them “filtered” the answer to correct for non-50/50 defectsdefects

• Two successive identical bits = 1 OR two successive different bits = 0Two successive identical bits = 1 OR two successive different bits = 0• Closer to 50/50 but (at least) twice as slowCloser to 50/50 but (at least) twice as slow

• Abandoned as too slow, too unwieldy, and—ultimately—Abandoned as too slow, too unwieldy, and—ultimately—undesirableundesirable

4

Pseudorandom generators (RNG)Pseudorandom generators (RNG)

• We cannot have (nor do we really want) truly random We cannot have (nor do we really want) truly random variables.variables.

• Any random number generator from a computer is not Any random number generator from a computer is not truly random (since it is repeatable).truly random (since it is repeatable).

• Referred to as “pseudorandom”Referred to as “pseudorandom”

• This repeatability is actually very important to us for This repeatability is actually very important to us for validation reasons. Also, this repeatability is the basis of validation reasons. Also, this repeatability is the basis of some Monte Carlo perturbation methods.some Monte Carlo perturbation methods.

• The most common form is that of "linear congruential The most common form is that of "linear congruential generators" which generate a series of bounded integers, generators" which generate a series of bounded integers, with each one using the one before it with the formula:with each one using the one before it with the formula:

m

ibaii n

nnn1

11

m) (mod

5

Possible difficultiesPossible difficulties

• Periodicity: Each number depends uniquely on the one before it. Periodicity: Each number depends uniquely on the one before it. So once a number repeats, the rest of the series will be the So once a number repeats, the rest of the series will be the same as it was before. How many unique numbers you get is same as it was before. How many unique numbers you get is the PERIOD.the PERIOD.

• It can be shown that a "full" period of m can be assured by It can be shown that a "full" period of m can be assured by choosing:choosing:

• a(mod 4)=1a(mod 4)=1• m as a power of 2m as a power of 2• b odd.b odd.

• The finiteness of the string also means that certain domains of The finiteness of the string also means that certain domains of the problem may be unreachable. (That is, there will be “gaps” the problem may be unreachable. (That is, there will be “gaps” of width 1/m.) [STARS]of width 1/m.) [STARS]

• We have to be careful not to depend on the later digits of a We have to be careful not to depend on the later digits of a uniform deviate: they may be less random than the early digits. uniform deviate: they may be less random than the early digits. (They may be the “remnants” of repeating digits of fractions.)(They may be the “remnants” of repeating digits of fractions.)

6

Example: m=8, a=5, b=3, i0=1Example: m=8, a=5, b=3, i0=1

## ii ai+bai+b mod mmod m

00 11 88 00 00

11 00 33 33 .375.375

22 33 1818 22 .25.25

33 22 1313 55 .625.625

44 55 2828 44 .5.5

55 44 2323 77 .875.875

66 77 3838 66 .75.75

77 66 3333 11 .125.125

7

Extending the periodExtending the period

• Even with “full period” L.C.G. random number generators, we still Even with “full period” L.C.G. random number generators, we still have the related problems:have the related problems:

• The period can be no longer than mThe period can be no longer than m• m must be a number that can be represented as an integer:m must be a number that can be represented as an integer:

where N is the number of bits used to store integers (usu. 16. 32, or 64)where N is the number of bits used to store integers (usu. 16. 32, or 64)

• Fortunately, we can solve this problem by combining the output of Fortunately, we can solve this problem by combining the output of several uncorrelated L.C.G.’s:several uncorrelated L.C.G.’s:

• The resulting period is the PRODUCT of the individual periodsThe resulting period is the PRODUCT of the individual periods• The following slide shows how KENO and MCNP implement thisThe following slide shows how KENO and MCNP implement this

12 Nm

N 21FRAC

8

KENO RNG (FORTRAN 77)KENO RNG (FORTRAN 77)

DOUBLE PRECISION FUNCTION FLTRN()DOUBLE PRECISION FUNCTION FLTRN()INTEGER X, Y, Z, X0, Y0, Z0, XM, YM, ZMINTEGER X, Y, Z, X0, Y0, Z0, XM, YM, ZMDOUBLE PRECISION V, X1, Y1, Z1DOUBLE PRECISION V, X1, Y1, Z1EXTERNAL RANDNUMEXTERNAL RANDNUMCOMMON /QRNDOM/ X, Y, ZCOMMON /QRNDOM/ X, Y, ZSAVE /QRNDOM/SAVE /QRNDOM/PARAMETER ( MX = 157, MY = 146, MZ = 142 )PARAMETER ( MX = 157, MY = 146, MZ = 142 )PARAMETER ( X0 = 32363, Y0 = 31727, Z0 = 31657 )PARAMETER ( X0 = 32363, Y0 = 31727, Z0 = 31657 )PARAMETER ( X1 = X0, Y1 = Y0, Z1 = Z0 )PARAMETER ( X1 = X0, Y1 = Y0, Z1 = Z0 )X = MOD(MX*X,X0)X = MOD(MX*X,X0)Y = MOD(MY*Y,Y0)Y = MOD(MY*Y,Y0)Z = MOD(MZ*Z,Z0)Z = MOD(MZ*Z,Z0)V = X/X1 + Y/Y1 + Z/Z1V = X/X1 + Y/Y1 + Z/Z1FLTRN = V - INT(V)FLTRN = V - INT(V)RETURNRETURNENDEND

9

MCNP5 RNG (FORTRAN 95)MCNP5 RNG (FORTRAN 95) integer, private, parameter :: R8 = selected_real_kind( 15, 307 )integer, private, parameter :: R8 = selected_real_kind( 15, 307 )

integer, private, parameter :: I8 = selected_int_kind( 18 ) integer, private, parameter :: I8 = selected_int_kind( 18 )

......

integer(I8), private, SAVE :: &integer(I8), private, SAVE :: &

& RN_MULT = 5_I8**19, && RN_MULT = 5_I8**19, &

& RN_ADD = 0_I8, && RN_ADD = 0_I8, &

& RN_STRIDE = 152917, && RN_STRIDE = 152917, &

& RN_SEED0 = 5_I8**19, && RN_SEED0 = 5_I8**19, &

& RN_MOD = 2_I8**48, && RN_MOD = 2_I8**48, &

& RN_MASK = 2_I8**48 - 1_I8, && RN_MASK = 2_I8**48 - 1_I8, &

& RN_PERIOD = 2_I8**46& RN_PERIOD = 2_I8**46

real(R8), private, SAVE :: &real(R8), private, SAVE :: &

& RN_NORM = 1.0_R8/2._R8**48& RN_NORM = 1.0_R8/2._R8**48

......

function rang()function rang()

implicit noneimplicit none

real(R8) :: rangreal(R8) :: rang

RN_SEED = iand( iand( RN_MULT*RN_SEED, RN_MASK) + RN_ADD, RN_MASK)RN_SEED = iand( iand( RN_MULT*RN_SEED, RN_MASK) + RN_ADD, RN_MASK)

rang = RN_SEED * RN_NORMrang = RN_SEED * RN_NORM

RN_COUNT = RN_COUNT + 1RN_COUNT = RN_COUNT + 1

returnreturn

end function rangend function rang

10

“Dimensionality”“Dimensionality”• Monte Carlo theory uses the word “dimension” in a Monte Carlo theory uses the word “dimension” in a

way that is a little foreign to normal usage, so be way that is a little foreign to normal usage, so be readyready

• The dimension of a Monte Carlo algorithm is how The dimension of a Monte Carlo algorithm is how many random numbers have to be drawn to many random numbers have to be drawn to generate one estimategenerate one estimate

• Examples:Examples:• Coin toss problem: 1 dimensionCoin toss problem: 1 dimension• Our pi problem: 2 dimensionOur pi problem: 2 dimension• Homework #1 problem of x1+x2>1.4: 2 dimensionHomework #1 problem of x1+x2>1.4: 2 dimension• Neutron transport problem simulation: ?Neutron transport problem simulation: ?

• Related to the “unit hypercube” point of viewRelated to the “unit hypercube” point of view

11

“Discrepancy”“Discrepancy”

• Monte Carlo theory also has the special word Monte Carlo theory also has the special word “discrepancy” which is important“discrepancy” which is important

• The discrepancy of a RNG is a measure of the biggest The discrepancy of a RNG is a measure of the biggest “hole” in a ordered series of random numbers“hole” in a ordered series of random numbers

• Usually defined (the so-called “star discrepancy”) as the Usually defined (the so-called “star discrepancy”) as the biggest difference between what fraction SHOULD have biggest difference between what fraction SHOULD have been chosen in a space from the origin (0,0,…,0) to any been chosen in a space from the origin (0,0,…,0) to any point (x,y,z,…) and the actual number that was point (x,y,z,…) and the actual number that was ACTUALLY chosen [blackboard example]ACTUALLY chosen [blackboard example]

• The fraction that should have fallen in is x*y*z…The fraction that should have fallen in is x*y*z…• You only have to check at actual chosen points (with and without You only have to check at actual chosen points (with and without

the point)the point)

• To my surprise, the size of the discrepancy is To my surprise, the size of the discrepancy is proportional toproportional to

• It is the reason for the slow convergence of Monte Carlo It is the reason for the slow convergence of Monte Carlo

1N

12

“Discrepancy”“Discrepancy”

13

Quasi-random numbersQuasi-random numbers

• The logical extension of the idea of stratified The logical extension of the idea of stratified sampling is to try to make N=Msampling is to try to make N=M

• Each of the i=1,2,…,N random numbers is located in its Each of the i=1,2,…,N random numbers is located in its own subdomain ((i-1)/N,i/N)own subdomain ((i-1)/N,i/N)

• This looks (and acts) in 1D like a simple Riemann choiceThis looks (and acts) in 1D like a simple Riemann choice• The trick is to make them jump around “randomly”The trick is to make them jump around “randomly”

• Result: Quasi-random numbers (also called “Low Result: Quasi-random numbers (also called “Low discrepancy sequences”)discrepancy sequences”)

• They are not really random, so they have to be used very They are not really random, so they have to be used very carefully. carefully.

• They are not used very much for Monte Carlo methods in They are not used very much for Monte Carlo methods in radiation transport. radiation transport.

• But, because they are of such importance to the But, because they are of such importance to the general field of Monte Carlo (i.e., beyond transport general field of Monte Carlo (i.e., beyond transport methods), you should have a taste of what they methods), you should have a taste of what they are, how to get and use them.are, how to get and use them.

14

Quasi-random numbers (2)Quasi-random numbers (2)

• "Quasi"-random numbers are not random at all; "Quasi"-random numbers are not random at all; they are a deterministic, equal-division, sequence they are a deterministic, equal-division, sequence that are "ordered" in such a way that they can be that are "ordered" in such a way that they can be used by Monte Carlo methods. used by Monte Carlo methods.

• The easiest to generate are “van der Corput” The easiest to generate are “van der Corput” sequences, which are found by "reflecting" the sequences, which are found by "reflecting" the digits of the base counting integers about their digits of the base counting integers about their radix point. radix point.

• You pick a base.You pick a base.• Count in that base.Count in that base.• Reflect the numberReflect the number• Interpret as base 10. Interpret as base 10.

15

Quasi-random numbers (3)Quasi-random numbers (3)

• After M=BaseAfter M=BaseNN-1 numbers in the sequence, the -1 numbers in the sequence, the sequence consists of the equally spaced fractions sequence consists of the equally spaced fractions (i/(M+1), i=1,2,3,...M). (i/(M+1), i=1,2,3,...M).

• Therefore, the sequence is actually deterministic, Therefore, the sequence is actually deterministic, not stochastic. (Like a numerical method.)not stochastic. (Like a numerical method.)

• The beauty of the sequence is that the re-ordering The beauty of the sequence is that the re-ordering results in the an (almost) even coverage of the results in the an (almost) even coverage of the domain (0,1) even if you stop in the middle.domain (0,1) even if you stop in the middle.

• Because you need ALL of the numbers in the Because you need ALL of the numbers in the sequence to cover the (0,1) range (which is not true sequence to cover the (0,1) range (which is not true of the pseudo-random sequence), it is important of the pseudo-random sequence), it is important that all of the numbers of the sequence be used on that all of the numbers of the sequence be used on the SAME decision. the SAME decision.

• So, if your problem has more than one decision (e.g., the So, if your problem has more than one decision (e.g., the pi problem had 2—x and y coordinates) you have to let pi problem had 2—x and y coordinates) you have to let each decision have its own RNGeach decision have its own RNG

16

Quasi-random numbers (4)Quasi-random numbers (4)• For a Monte Carlo process that has more than one For a Monte Carlo process that has more than one

decision, a different sequence must be used for decision, a different sequence must be used for each decision. (This is why we don’t use them.)each decision. (This is why we don’t use them.)

• The most common way this situation is handled is The most common way this situation is handled is to use prime numbers in order—2,3,5,7,11,etc.—for to use prime numbers in order—2,3,5,7,11,etc.—for decisions. (“Halton” sequence, after UNC decisions. (“Halton” sequence, after UNC professor)professor)

• Asymptotic estimate of error is which Asymptotic estimate of error is which means closer to ~ 1/N means closer to ~ 1/N

• but deteriorates very quickly with dimensionalitybut deteriorates very quickly with dimensionality

• A Java subroutine is given in the public area.A Java subroutine is given in the public area.• The resulting standard deviations calculated and The resulting standard deviations calculated and

printed by standard MC codes are NOT accurateprinted by standard MC codes are NOT accurate

N

N d 110 )(log

17

Overview of pdf and cdfOverview of pdf and cdf

• Basic definition of probability distribution Basic definition of probability distribution function (p.d.f.):function (p.d.f.):

• And its integral, the cumulative distribution And its integral, the cumulative distribution function (c.d.f.):function (c.d.f.):

})d,(in lies Pr{)( xxxxdxx i

} Pr{')'( xxdxxx i

x

18

Overview of pdf and cdf (2)Overview of pdf and cdf (2)

• Corollaries of these definitions:Corollaries of these definitions:

000.1')'(

} Pr{')'(

} Pr{')'(

dxx

bxadxxab

xxdxxx

i

b

a

i

x

19

Mapping ->x using p(x)Mapping ->x using p(x)

• Our basic technique is to use a unique Our basic technique is to use a unique y->xy->x• y=y= from (0,1) and x from (a,b) from (0,1) and x from (a,b)• We are going to use the mapping backwardsWe are going to use the mapping backwards

20

Mapping (2)Mapping (2)

Note that:Note that:• (a)=0(a)=0• (b)=1(b)=1• Function is non-decreasing over domain (a,b)Function is non-decreasing over domain (a,b)

Our problem reduces to:Our problem reduces to:• Finding Finding (x)(x)• Inverting to get x(Inverting to get x(), a formula for turning pseudo-), a formula for turning pseudo-

random numbers into numbers distributed according random numbers into numbers distributed according to desired to desired (x)(x)

21

Mapping (3)Mapping (3)

• We must have:We must have:

(x)xdxxx

dxxax

xdx

xd

dxxd

xxxx

x

a

x

a

of C.D.F the,)(')'()(

')'()()(

)()(

)(

})d,(in lies Pr{})d,(in lies Pr{

22

Resulting general procedureResulting general procedure

• Form CDF:Form CDF:

• Set equal to pseudo-random number:Set equal to pseudo-random number:

• Invert to get formula that translates from Invert to get formula that translates from to x: to x:

x

a

dxxx ')'()(

)(x

)(1 x

23

Uniform distributionUniform distribution

• For our first distribution, pick x uniformly in range For our first distribution, pick x uniformly in range (a,b):(a,b):

• Step 1: Form CDF.Step 1: Form CDF.

abdxdxx

xx

baxx

b

a

b

a

1

1

1

)(~

)(~)(

, ,1)(~

ab

axdxab

dxxxx

a

x

a

'1

')'()(

24

Uniform distribution (2)Uniform distribution (2)

• Step 2: Set pseudo-random number to CDF:Step 2: Set pseudo-random number to CDF:

• Step 3: Invert to get Step 3: Invert to get x(x())::

• Example: Choose Example: Choose uniformly in (-1,1): uniformly in (-1,1):

ab

ax

)( abax

12))1(1(1

25

Discrete distributionDiscrete distribution

• For a discrete distribution, we have N For a discrete distribution, we have N choices of state choices of state ii, each with probability , , each with probability , so:so:

• Step 1: Form CDF:Step 1: Form CDF:

i

1 1 2 2( ) ( ) ( ) ( )N Nx x x x

N

iNi

x

xx

xxx

xxx

xx

dxxx

1

3221

211

1

,1

,

,

,0

')'()(

26

Discrete distribution (2)Discrete distribution (2)

• Step 2: Set pseudo-random number to Step 2: Set pseudo-random number to CDF:CDF:

• Step 3: Invert to get Step 3: Invert to get x(x())::)(x

1 if ,

if ,

if ,

0 if ,

1

1

1

1

1

2112

11

1

ξx

ξx

x

x

x

N-

iiN

n

ii

n-

iin

27

Discrete distribution (3)Discrete distribution (3)

• Example: Choose among 3 states with Example: Choose among 3 states with relative probabilities of 4, 5, and 6.relative probabilities of 4, 5, and 6.

28

Continuous distribution: DirectContinuous distribution: Direct

• This fits the “pure” form developed before.This fits the “pure” form developed before.• Form CDF:Form CDF:

• Set equal to pseudo-random number:Set equal to pseudo-random number:

• Invert to get formula that translates from Invert to get formula that translates from to x: to x:

x

a

dxxx ')'()(

)(x

)(1 x

29

Continuous: Direct (2)Continuous: Direct (2)

• Example: Pick x from:Example: Pick x from:

xex x 0,)(~

30

Homework from textHomework from text

31

Homework from textHomework from text

32

Homework from textHomework from text

33

Homework from textHomework from text

34

Homework from textHomework from text

35

Homework from textHomework from text