par_rand_num_gen

8/7/2019 par_rand_num_gen

http://slidepdf.com/reader/full/parrandnumgen 1/37

PARALLEL RANDOM

NUMBER

GENERATION

Ahmet Duran

CISC 879



Outline

j Random Numbers

± Why To Prefer Pseudo- Random Numbers

± Application Areas

± The Linear Congruential Generators ± Period of Linear Congruential Sequence

j Approaches to the Generation of RandomNumbers on Parallel Computers

± Centralized ± Replicated

± Distributed



Outline

jSome Tests For Random Number

Generators

jTesting Parallel Random NumberGenerators

jSPRNG (Scalable Library for Pseudorandom

Number Generation)



Random Numbers

jWell-known sequential techniques exist for

generating, in a deterministic fashion,

j The number sequences are largely

indistinguishable from true random sequences

j The deterministic nature is important because

it provides for reproducibility in computations.

j Efficiency is needed by preserving randomnessand reproducibility



Random Numbers

j Consider the following experiment to verif y therandomness of an infinite sequence of integersin [1,d].

± Suppose we let you view as many numbers from thesequence as you wished to.

± You should then guess any other number in thesequence.

± If the likelihood of your guess being correct is

greater than 1/d, then the sequence is not random. ± In practice, to estimate the winning probabilities we

must play this guessing game several times.



Why To Prefer Pseudo-

Random Numbersj True random numbers are rarely used in

computing because:

± difficult to generate reliably

± the lack of reproducibility would make thevalidation of programs that use them extremely difficult

j Computers use pseudo-random numbers:

± finite sequences generated by a deterministicprocess

± but indistinguishable, by some set of statistical tests,from a random sequence.



Application Areas

j Simulation: When we simulate naturalphenomena, random numbers are required tomake things realistic. For example:

Operations research (where people come intoairport at random intervals.)

j Sampling: It is often impractical to examineall possible cases, but a random sample will

provide insight into what constitutes "typical"behaviour.

j Computer Programming: Random valuesmake good source of data for testing theeffectiveness of computer algorithms.



Application Areas

j Numerical Analysis

j Decision Making: There are reports that many executives make their decisions by flipping a

coin or by throwing darts, etc.j Aesthetics: A little bit randomness makes

computer-generated graphics and music seemmore lively.

j Recreation: "Monte Carlo method", a generalterm used to describe any algorithm thatemploys random numbers.



The Linear Congruential

Generatorsj Generator is a function, when applied to a

number, yields the next number in the

sequence. For example,

j X_k+1 = (a * X_k + c) mod m

where X_k is the k th element of the sequence and

X_0, a, c , and m define the generator.

j

As numbers are taken from a finite set (forexample, integers between 1 and 2^31), any

generator will eventually repeat itself.



The Linear Congruential

GeneratorsjThe length of the repeated cycle is called

the period of the generator.

jA good generator is one with a longperiod and no discernible correlation

between elements of the sequence.

jExample: X_k+1 = (3 * X_k + 4) mod 8

E = {1, 7, 1, 7, «} is bad.



Period of Linear Congruential

SequencejTheorem: The linear congruential

sequence has period m if and only if

± c is relatively prime to m;

± b = a - 1 is a multiple of p, for every prime pdividing m;

± b is a multiple of 4, if m is a multiple of 4.

j

Example: X_k+1 = (7 * X_k + 5) mod 18E = {1, 12, 17, 16, 9, 14, 13, 6, 11, 10, 3, 8,7, 0, 5, 4, 15, 2, 1,«}



Approaches to the Generation

of Random Numbers on ParallelComputers

jCentralized

jReplicated

jDistributed



Centralized Approach

j A sequential generator is encapsulated in atask from which other tasks request randomnumbers.

j Advantages: ± Avoids the problem of generating multiple

independent random sequences

j Disadvantages:

± Unlikely to provide good performance ± Makes reproducibility hard to achieve: the

response to a request depends on when it arrives atthe generator, and the result computed by aprogram can vary from one run to the next.



Replicated Approach

j Multiple instances of the same generator arecreated (for example, one per task). Each generator uses either the same seed or a unique

seed, derived, for example, from a task identifier.

j Advantages:

± Efficiency

± Ease of implementationj Disadvantages:

± Not guaranteed to be independent and, can sufferfrom serious correlation problems.



Distributed Approach

j Responsibility for generating a single sequence

is partitioned among many generators, which

can then be parcelled out to different tasks.

j Advantages:

± The analysis of the statistical properties of the

distributed generator is simplified because the

generators are all derived from a single generator

± Efficiency

j Disadvantages:

± Difficult to implement on parallel computers



Distributed Approaches

j The techniques described here are based on

an adaptation of the linear congruential

algorithm called the random tree method.

j This facility is particularly valuable in

computations that create and destroy tasks

dynamically during program execution.

j The Random Tree Method

j The Leapfrog Method

j Modified Leapfrog



The Random Tree Method

jThe random tree method employs two

linear congruential generators, L and R

, that differ only in the values used for a .

jL_k+1 = a_L L_k mod m

jR_k+1 = a_R R_k mod m

R1

L

R0

R2



j Application of the left generator L to a seed

generates one random sequence; applicationof the right generator R to the same seed

generates a different sequence.

j By applying the right generator to elements of

the left generator's sequence (or vice versa), atree of random numbers can be generated.

j By convention, the right generator R is used

to generate random values for use in

computation, while the left generator L is

applied to values computed by R to obtain the

starting points R0, R1 , etc., for new right

sequences.



Random Tree Method

j Advantages:

± Useful to generate new random sequences in a

reproducible and noncentralized fashion. This is

valuable, in applications in which new tasks andhence new random generators must be created

dynamically.

j Disadvantages:

± There is no guarantee that different right sequenceswill not overlap. If two starting points happen to be

close to each other, the two right sequences that are

generated will be highly correlated.



The Leapfrog Method

jA variant of the random tree methodjUsed to generate sequences that can be

guaranteed not to overlap for a certain

period.jUseful where a program requires a fixed

number of generators. (For example, one

generator for each task ).



L_0

L_1L_2

L_3

L_4

L_5L_6

L_7

L_8

R0 R1 R2

Figure: The leapfrog method with n=3. Each of the right

generators selects a disjoint subsequence of the sequence

constructed by the left generator¶s sequence.



j Let n be the number of sequences required. Then we

define a_L and a_R as a and a^n, respectively,

j L_k+1 = a L_k mod mj R_k+1 = a^n R_k mod m

j Create n different right generators R0 .. Rn-1 by taking

the first n elements of L as their starting values.

j The name ³leapfrog method´ refers that the i thsequence Ri consists of L_i and every n th subsequent

element of the sequence generated by L.

j As the method partitions the elements of L, each

subsequence has a period of at least P/n, where P is the

period of L.

j The n subsequences are disjoint for their first P/n

elements.



j The generator for the r th subsequence, Rr , is defined

by a^n and Rr_0 = L_r.

j First, compute a^r and a^nj Then, compute members of the sequence Rr as follows,

to obtain n generators, each defined by a triple (Rr_0,

a^n, m) , for

0 <= r < n .j Rr_0 = (a^r L_0) mod m

j Rr_i+1 = (a^n Rr_I) mod m



Modified Leapfrog

j A variant of the leapfrog method

j Used in situations, where we know the maximum

number, n , of random values needed in a subsequence

but not the number of subsequences required.j The role of L and R are reversed so that the elements of

subsequence i are the contiguous elements

j L_k+1 = a^n L_k mod m

j

R_k+1 = a R_k mod mj It is not a good idea to choose n as a power of two, as

this can lead to serious long-term correlations.



L_0

L_1L_2

L_3

L_4

L_5L_6

L_7

L_8

R0 R1 R2 «

F igure: Modified l eapfrog wi th n=3 . E ach subsequence

cont ain

s three

con

t iguou

snumber

s from

the main

se uence .



Some Tests For Random

Number Generatorsj The basic idea behind the statistical tests is that the

rabdom number streams obtained from a generator

should have the properties of a random sample drawn

from the uniform distribution.

j Tests are designed so that the expected value of some

test statistic is known for uniform distribution.

j The empirically generated random number stream is

then subject to the same test, and the statistic obtained

is compared against the expected value.



Frequency Test

j The focus of the test is the proportion of zeroesand ones for the entire sequence.

j Example:

± (input)

E=110010010000111111011010101000100010000101

10100011000010001101001100010011000110011000

10100010111000

± (input) n = 100

± (output) P-value = 0.109599

± (conclusion) Since P-value >= 0.01, accept the

sequence as random.



Runs Test

j A run is an uninterrupted sequence of identicalbits.

j The focus of this test is the total number of runsin the sequence.

j A run of length k consists of exactly k identicalbits and is bounded before and after with a bitof the opposite value.

j The purpose of the runs test is to determine

whether the number of runs of ones and zerosof various lengths is as expected for a randomsequence.

j Determines whether the oscillation between

such zeros and ones is too fast or too slow.



Runs Test

j A fast oscillation occurs when there are a lot of

changes, e.g., 010101010 oscillates with every bit.

j (input) E =

1100100100001111110110101010001000100001011010001100001000110100110001001100011001

100010100010111000

j (input) n = 100

j (output) P-value = 0.500798

j (conclusion) Since P-value >= 0.01, accept the

sequence as random.



Testing Parallel Random

Number GeneratorsjA good parallel random number

generator must be a good sequential

generator.

jSequential tests check for correlations

within a stream, while parallel tests check

for correlations between different

streams.



Testing Parallel Random

Number GeneratorsjExponential sums

jParallel spectral test

j Interleaved testsjFourier transform test

jBlocking test



Fourier Transform Test

jFill a two dimensional array with randomnumbers. Each row of the array is filledwith random numbers from a different

stream.jCalculate the Fourier coefficients and

compare with the expected values.

j

This test is repeated several times andcheck if there are particular coefficientsthat are repeatedly ³bad´.



Blocking Test

jUse the fact that the sum of independentvariables asymptotically approaches thenormal distribution to test for the

independence of random numberstreams.

jAdd random numbers from severalstream and form a sum.

jGenerate several such sums and check if their distribution is normal.



SPRNG

j A ³Scalable Library for Pseudorandom Number

Generation´,

j Designed to use parametrized pseudorandom number

generators to provide random number streams toparallel processes.

j Includes

± Several, qualitatively distinct, well tested, scalable

RNGs

± Initialization without interprocessor communication

± Reproducibility by using the parameters to index the

streams



± Reproducibility controlled by a single ³global´ seed

± Minimization of interprocessor correlation with the

included generators

± A uniform C, C++, Fortran and MPI interface ± Extensibility

± An integrated test suite.



Refences:

j Introduction to Parallel RNGs

http://sprng.cs.fsu.edu/Version1.0/paper/index.html

j Random Number Generation on Parallel

Computer Systems

http://www.npac.syr.edu/projects/reu/reu94/cstoner

/proposal/proposal.html

j SPRNG (Scalable Parallel Pseudo Random

Number Generators Library)

http://sprng.cs.fsu.edu/



References (continued)

j A. Srinivasan, D. Ceperley and M. Mascagni,Testing Parallel Random Number Generators

http://sprng.cs.fsu.edu/links.html

j

M. Mascagni, D. Ceperley and A. Srinivasan,SPRNG: A Scalable Library for PseudorandomNumber Generation

http://sprng.cs.fsu.edu/links.html

j Ian Foster, Designing and Building ParallelPrograms

j D. E. Knuth, The Art of ComputerProgramming, Volume 2, SeminumericalAlgorithms, Third Edition

par_rand_num_gen

Documents

Transcript of par_rand_num_gen