Monte` Carlo Methods 1 MONTE` CARLO METHODS INTEGRATION and SAMPLING TECHNIQUES.
Chapter 4: Monte Carlo Methods -...
Transcript of Chapter 4: Monte Carlo Methods -...
Chapter 4: Monte Carlo Methods
Paisan Nakmahachalasint
Introduction
Monte Carlo Methods are a class of computational algorithms that rely on repeated random sampling to compute their results.Monte Carlo simulation methods are especially useful in studying systems with a large number of coupled degrees of freedom.
Introduction
Monte Carlo methods describe a large and widely-used class of approaches which tend to follow a particular pattern:
Define a domain of possible inputs. Generate inputs randomly from the domain using a certain specified probability distribution.
Perform a deterministic computation using the inputs. Aggregate the results of the individual computations into the final result.
Uniform Random Number Generators
Monte Carlo simulations are inherently probabilistic, and thus make frequent use of programs to generate random numbers.
On a computer these numbers are not random at all -- they are strictly deterministic and reproducible, but they look like a stream of random numbers. For this reason such programs are more correctly called pseudo-random number generators.
Uniform Random Number Generators
Essential Properties
Repeatability -- the same sequence should be produced with the same seeds.Randomness -- should produce independent uniformly distributed random variables that pass all statistical tests for randomness. Long period – The period should be much longer than the amount of random numbers needed.Insensitive to seeds -- period and randomness properties should not depend on the initial seeds.
Uniform Random Number Generators
Additional PropertiesPortability -- should give the same results on different computers. Efficiency -- should be fast (small number of floating point operations) and not use much memory. Disjoint subsequences -- different seeds should produce long independent (disjoint) subsequences so that there are no correlations between simulations with different initial seeds. Homogeneity -- sequences of all bits should be random.
Uniform Random Number Generators
Middle-Square Method
Start with a four-digit number x0, called the seed.
Square it to obtain an eight-digit number(add a leading zero if necessary).Take the middle four digits as the next random number.
64801004104901801009742316562041xn
876543210n
Uniform Random Number Generators
Linear Congruence Generator (LCG)
Introduced by D. H. Lehmer in 1951. A great majority of pseudo-random numbers used today are based on this method.
xn+1 = (axn + b) (mod m)
Example: if x0 = 7, a = 1, b = 7, m = 10, we obtain the following sequence:
7, 4, 1, 8, 5, 2, 9, 6, 3, 0, 7, 4, …
Uniform Random Number Generators
Choice of the modulus, m.m should be as large as possible since the period can never be larger than m.One usually chooses m to be near the largest integer that can be represented.
Choice of the multiplier, aIt was proven by M.Greenberger in 1961 that the sequence will have period m if and only if
b is a relative prime to m.a – 1 is a multiple of p for all p | m.4|m ⇒ 4|a – 1.
Uniform Random Number Generators
With b = 0, one cannot get the full period, but in order to get the maximum possible, the followings should be satisfied:
x0 is a relative prime to m.a is a primitive root modulo m.
It is possible to obtain a period of length m – 1, but usually the period is around m/4.
Uniform Random Number Generators
A Disadvantage of LCGRANDU is a popular random number generator distributed by IBM in 1960’s with the algorithm
xn+1 = 65539xn (mod 231).This generator was later found to have a serious problem – all random numbers fall into hyperplanes.
Uniform Random Number Generators
5000 random points in 1 and 2 dimensions generated from RANDU.
0.2 0.4 0.6 0.8 1.
10
20
30
40
50
60
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
Histogram of 5000random numbers.
Distribution of 5000 points in 2 dimensions
Uniform Random Number Generators
5000 random points in 3 dimensions generated from RANDU viewed at two different view points.
Uniform Random Number Generators
LCGs in rand() in various compilers.
In all above compilers, m = 232
16 - 30123451103515245ANSI C
32 - 631134775813Borland Delphi
16 - 302531011214013Microsoft C/C++
0 - 30123451103515245Glibc (GCC)
16 - 30122695477Borland C/C++
Output bits in rand()baSource
Uniform Random Number Generators
Lagged-Fibonacci Generator (LFG)
Lagged Fibonacci pseudo-random number generators have become increasingly popular in recent years. These generators are so named because of their similarity to the familiar Fibonacci sequence. LFG uses the algorithm
xn = xn – l + xn – k (mod m) where l > k > 0
with m initial values {x0, x1, …, xl – 1}.
Uniform Random Number Generators
For most applications of interest, m is a power of two; i.e., m = 2M.
With a proper choice of l, k and the initial values, the period P of LFG is (2l – 1)×2M – 1.
Two commonly used LFGs.
xn = xn – 17 + xn – 5 (mod 231) period ≈ 247
xn = xn – 55 + xn – 24 (mod 231) period ≈ 285
Uniform Random Number Generators
Unlike LCG, the value of the modulus, m, does not by itself limit the period of the generator.LFG is computationally simple: an integer add, a logical AND (to accomplish the mod 2M operation), and the advancing of two array pointers are the only operations required to produce a new random number.
Random Variables and Distribution
A random variable can be thought of as an unknown value that may change every time it is inspected. Thus, a random variable can be thought of as a function mapping the sample space of a random process to the real numbers.
A random variable can be discrete or continuous.
Random Variables and Distribution
Example (Discrete RV)The sample space of tossing 2 fair coins is
S = {HH, HT, TH, TT}.
Let X = number of heads, then
X = {(HH,2), (HT,1), (TH,1), (TT,0)}.
Random Variables and Distribution
Example (Cont’d)The probability function
of X is given by
The mean of X is
The variance of X is
141214
if 0
if 1( )
if 2
0 otherwise
x
xP X x
x
⎧⎪ =⎪⎪⎪ =⎪⎪= = ⎨⎪ =⎪⎪⎪⎪⎪⎩
1 1 14 2 4
E[ ] ( ) 0 1 2 1X i i
i
X x P xμ = = = ⋅ + ⋅ + ⋅ =∑
2 2 2 12
Var[ ] E[( ) ] E[ ]X X
X X Xμ μ= − = − =
Random Variables and Distribution
Example: Binomial Distribution X ~ B(n,p)
( )( ) 1 0,1, ,n xxn
P X x p p x nx−⎛ ⎞⎟⎜ ⎟= = − =⎜ ⎟⎜ ⎟⎜⎝ ⎠
…
( ) ( )0
22
0
E[ ] ( )
Var[ ] ( ) 1
n
xn
x
X xP x np
X x P x np np p
=
=
= =
⎛ ⎞⎟⎜ ⎟= − = −⎜ ⎟⎜ ⎟⎜⎝ ⎠
∑
∑
Random Variables and Distribution
Example: Continuous Uniform DistributionX ~ U(a,b)
1 for( )
0 otherwiseb a
X
a x bf x −
⎧⎪ ≤ ≤⎪= ⎨⎪⎪⎩
( )
( )2
22
12
112
E[ ] ( )
Var[ ] ( )2
X
X
X xf x dx a b
a bX x f x dx b a
∞
−∞
∞
−∞
= = +
⎛ ⎞⎛ ⎞ + ⎟⎜⎟⎜ ⎟= − = −⎜⎟⎜ ⎟⎟ ⎜⎜ ⎟⎜⎝ ⎠ ⎝ ⎠
∫
∫
Random Variables and Distribution
Cumulative Distribution Function
Example( ) ( ) ( )
x
X XF x P X x f t dt
−∞= ≤ = ∫
0
( ) for
1
x aX b a
x a
F x a x b
x b
−−
⎧⎪ <⎪⎪⎪= ≤ ≤⎨⎪⎪ >⎪⎪⎩
1 for( )
0 otherwiseb a
X
a x bf x −
⎧⎪ ≤ ≤⎪= ⎨⎪⎪⎩
-1 0 1 2 3 4 5
0.2
0.4
0.6
0.8
1.0
-1 0 1 2 3 4 5
0.2
0.4
0.6
0.8
1.0
Random Variables and Distribution
Example: X ~ U(0,1) and Y = X2.
( )2
0 if 0
( ) if 0 1
1 if 1Y
y
F y P X y y y
y
⎧⎪ <⎪⎪⎪⎪= ≤ = ≤ ≤⎨⎪⎪ >⎪⎪⎪⎩
1
2if 0 1( )
( )0 otherwise
Y yY
ydF yf y
y
⎧⎪ ≤ ≤⎪⎪= = ⎨⎪⎪⎪⎩-0.5 0.0 0.5 1.0 1.5
1
2
3
4
Random Variables and Distribution
Example: X ~ U(0,1) and
( )212
14
0 if 0
( ) ( ) 2 if 0 1
1 ifY
y
F y P X y y y
y
⎧⎪ <⎪⎪⎪⎪= − ≤ = ≤ ≤⎨⎪⎪ >⎪⎪⎪⎩
1 14
if 0( )( )
0 otherwiseY y
Y
ydF yf y
y
⎧⎪ ≤ ≤⎪⎪= = ⎨⎪⎪⎪⎩
( )212
Y X= −
0.0 0.2 0.4 0.6 0.8 1.0
2
4
6
8
Random Variables and Distribution
Example: X ~ U(0,1), Y ~ U(0,1), Z = X+Y.
( )2
22
(2 )
2
0 if 0
if 0 1( )
1 if 1 2
1 if 2
z
Z z
z
zF z P X Y z
z
z
−
⎧⎪ <⎪⎪⎪ ≤ ≤⎪⎪= + ≤ = ⎨⎪ − < ≤⎪⎪⎪ >⎪⎪⎩
1 1 if 0 2( )( )
0 otherwiseZ
Z
z zdF zf z
z
⎧⎪ − − ≤ ≤⎪⎪= = ⎨⎪⎪⎪⎩
-0.5 0.0 0.5 1.0 1.5 2.0 2.5
0.2
0.4
0.6
0.8
1.0
Non-uniform Random Numbers
Inverse CDF MethodIf X is a random variable having the probability density function f X(x) and the cumulative distribution function (CDF)
then the random variable U = FX(X)is uniformly distributed on the interval [0,1].
Thus has the desired distribution.
( ) ( )x
X XF x f t dt
−∞= ∫
1( )X
X F U−=
Non-uniform Random Numbers
Example: An exponential distribution
for 0( )
0 for 0
x
X
e xf x
x
λλ −⎧⎪ >⎪⎪= ⎨⎪ ≤⎪⎪⎩
0 if 0
( ) 1 if 0 1
1 if 1
xX
x
F x e x
x
λ−
⎧⎪ <⎪⎪⎪= − ≤ ≤⎨⎪⎪ >⎪⎪⎩
lnwhere U(0,1)
UX U
λ= − ∼
0.0 0.5 1.0 1.5 2.0 2.5 3.0
0.5
1.0
1.5
2.0
0.0 0.5 1.0 1.5 2.0 2.5 3.0
0.20.40.60.81.0
Non-uniform Random Numbers
Example: A triangular distribution
2 2 4 for 0 1( )
0 otherwiseX
x xf x
⎧⎪ − − ≤ ≤⎪⎪= ⎨⎪⎪⎪⎩
2 12
2 12
0 if 0
2 if 0( )
1 2(1 ) if 1
1 if 1
X
x
x xF x
x x
x
⎧⎪ <⎪⎪⎪ ≤ ≤⎪⎪= ⎨⎪ − − < ≤⎪⎪⎪ >⎪⎪⎩
12 2
1 12 2
ifwhere U(0,1)
1 if
U
U
UX U
U−
⎧⎪ ≤⎪⎪= ⎨⎪ − >⎪⎪⎩
∼
-0.5 0.5 1.0 1.5
0.5
1.0
1.5
2.0
-0.5 0.5 1.0 1.5
0.20.40.60.81.0
Non-uniform Random Numbers
Simulated Result: A triangular distribution3N = 10
4N = 10
5N = 10 6N = 10
Non-uniform Random Numbers
Acceptance/Rejection MethodX is a random variable with pdf fX(x). We want to generate a random variable Ywith pdf fY(x) and there is a constant c for which ( ) ( ).
Y Xf x cf x≤
( )Yf x
( )X
cf x
x
Non-uniform Random Numbers
Acceptance/Rejection AlgorithmGenerate a random value x from X. Generate a random value u from U(0,1).
If , then x is accepted.
Otherwise, x is rejected (and repeat with the step of the generation of x).
( )
( )Y
X
f xu
cf x≤
Non-uniform Random Numbers
Box-Muller Method for Normal DistributionLet U ~ U(0,1) andgenerate u1 and u2 from U. Then
are independently distributed as the standard normal distribution.
1 1 2
2 1 2
cos(2 ) 2 ln
sin(2 ) 2 ln
x u u
x u u
π
π
= −
= −
Central Limit Theorem
Central Limit Theorem (CLT)
The sum of a sufficiently large number of independent and identically distributed (i.i.d.) random variables, each with finite mean and variance, will be approximately normally distributed.
Central Limit Theorem
Let X1, X2, ..., Xn be a sequence of n i.i.d. random variables each having finite values of expectation μ and variance σ2 > 0.
Let Sn = X1 + X2 + … + Xn. If we define
then Zn → N(0,1).
nn
X nZ
nμ
σ−
=
Central Limit Theorem
Example: If a fair coin is tossed 106 times, and we let X = number of heads, then
E[ ] 500, 000
Var[ ] (1 ) 250, 000
X np
X np p
= =
= − =500, 000
(0,1)500
XN
−→
499 000 500 000 501000 502000
0.0002
0.0004
0.0006
0.0008
Monte Carlo Simulation
Monte Carlo simulation was named for Monte Carlo, Monaco, where the primary attractions are casinos containing games of chance. Games of chance such as roulette wheels, dice, and slot machines, exhibit random behavior. Monte Carlo simulation randomlygenerates values for uncertain variables over and over to simulate a model.
Monte Carlo Simulation
Why Simulation?In some circumstances, it may not be feasible to observe the behavior directly or to conduct experiment.
A system of elevators during rush hour.
Automobile traffic controlling.
In other situations, the system for which alternative procedures need to be tested may not even exist yet.
Communication networks in an office buildingLocations of machines in a new industrial plant.
Monte Carlo Simulation
In such instances, the modeler might simulate the behavior and test the various alternatives being considered to estimate how each affects the behavior.
Build a scaled model.Drag force on a proposed submarine.Designs of a jet airplane.
Monte Carlo simulation which is typically accomplished with the aid of a computer.
Monte Carlo Simulation
Processes with an element of chance involved are called probabilistic, as opposed to deterministic.
Monte Carlo simulation is a probabilistic model. It can also be used to approximate a deterministic behavior.
Behavior
Probabilistic
Deterministic
Model
Probabilistic
Deterministic
Monte Carlo Simulation
Probabilistic Behavior: an Unfair DieConsider a probability model where each event is not equally likely. Assume a die is biased according to the following distribution.
0.16
0.25
0.34
0.23
0.12
0.11
P(roll)Roll value
SIX(0.9, 1.0]
FIVE(0.7, 0.9]
FOUR(0.4, 0.7]
THREE(0.2, 0.4]
TWO(0.1, 0.2]
ONE[0, 0.1]
AssignmentValue of xi
Monte Carlo Simulation
0.10.10450.10440.1040.1200.1506
0.20.20110.20120.2010.1840.2805
0.30.30810.30820.3080.3200.3504
0.20.19620.19620.1920.1990.2103
0.10.09920.09920.0990.0990.0802
0.10.09480.09480.0940.0980.1301
Expected Results400001000050001000100Die
Value
Monte Carlo Integration
Let p be the probability of a die having a desired value.Let N be the number of total tosses,and n be the number of tosses with the desired value.
E[ ]
Var[ ] (1 )
n pN
n p p N
== −
Monte Carlo Integration
If then
Example: If p = 0.1, then
( )2
1ˆE[ ] E[ ]
11ˆVar[ ] Var[ ]
p n pN
p pp n
NN
= =
−= =
( )ˆ
1 0.3ˆVar[ ]
p
p pp
N Nσ
−= = =
ˆ ,n
pN
=
Monte Carlo Integration
Area under a curve
area under curve number of points counted below curvearea of rectangle total number of random points
≈
Monte Carlo Integration
The area under the curve y = cos x over the interval -π/2 ≤ x ≤ π/2.
2.01186300001.96641000
2.01093200001.99666900
2.00978150001.99491800
2.00873100002.02857700
2.0066980002.09440600
2.0231960002.04832500
2.0143950002.12058400
1.9996240002.01064300
1.9771130002.13628200
1.9446520002.07345100
Approximation to Area
Number of points
Approximation to area
Number of points
Monte Carlo Integration
Let the area under the curve be I, while the area of the rectangle is R. Suppose that n out of N points fall under the curve, then we estimate the integral to be
There is a probability I/R that each random point will fall under the curve.
ˆ nI R
N= ⋅
Monte Carlo Integration
Thus, the number of random points fall under the curve will be binomially distributed.
E[ ]
Var[ ] 1
In N
RI I
n NR R
= ⋅
⎛ ⎞⎟⎜ ⎟= ⋅ −⎜ ⎟⎜ ⎟⎜⎝ ⎠
Monte Carlo Integration
Example: For
( )2
ˆE[ ] E[ ]
ˆVar[ ] Var[ ]
RI n I
NI R IR
I nN N
= =
⎛ ⎞ −⎟⎜ ⎟= =⎜ ⎟⎜ ⎟⎜⎝ ⎠
/2
/2cos , 2, .x dx I R
π
ππ
−= =∫
( )ˆ
2 2 1.5ˆVar[ ]I
IN N
πσ
−= = ≈
Monte Carlo Optimization
Powerful applications of Monte Carlo simulation lie in numerical optimization.There are many Monte Carlo optimization methods including Genetic Algorithm and Simulated Annealing.
The method is relatively simple, but can effectively solve many famous complex problems.
Monte Carlo Optimization
Simulated AnnealingSimulated annealing is an analogy with thermodynamics when liquids freeze and crystallize, or metals cool and anneal.At high temperatures, the molecules of liquids move freely. If the liquid is cooled slowly, thermal mobility is lost. Atoms line up to form a pure crystal, which is the state of minimum energy.
Monte Carlo Optimization
If the liquid is cooled quickly or quenched, it will not reach the ground state but ends up in some other states having somewhat higher energy.The essence of the process is slowcooling, allowing ample time for redistribution of the atoms as they lose mobility. This is the technical definition of annealing.
Monte Carlo Optimization
A system in thermal equilibrium at temperature T has its energy probabilistically distributed among all different energy states E according to the Boltzman distribution:
where k is the Boltzman’s constant.
/( ) E kTP E e−∼
Monte Carlo Optimization
At low temperatures, the system can still be in a high energy state with a small probability. This allow the system to get out of a local minimum in favor of finding a better, more global, one.
As the temperature is lowered, the system will be less likely to be in a higher energy state.
Monte Carlo Optimization
In 1953, Metropolis et al propose that a system will change its configuration from energy E1 to energy E2 with probability
This general scheme is known as the Metropolis algorithm.
2 1( )/2 1
1 21 2
if ( )
1 if
E E kTe E EP E E
E E
− −⎧⎪ ≥⎪⎪→ = ⎨⎪ <⎪⎪⎩
Monte Carlo Optimization
To make use of the Metropolis algorithm in general systems, we must provide:
Possible system configurations.
Random changes in the configuration
An objective function E to be minimized.
A control parameter T and an annealing schedule.
Could be accomplished with a physical insight or an trial-and-error experiments.
Monte Carlo Optimization
The Traveling Salesman ProblemA salesman visits N cities with given positions (xi,yi), finally returning to the city of origin. Each city is to be visited only once, and the route is to be made as short as possible.
The cities are number i = 1, 2, …, N.
Monte Carlo Optimization
Configuration: A configuration is the order in which the cities are visited; thus, it is a permutation of 1, 2, …, N.Rearrangments: An efficient suggested moves are done by
Reversal: A section of path is removed and replaced with the same cities in opposite order.
Transport: A section of path is removed and replaced in between two cities on another randomly chosen part of the path.
Monte Carlo Optimization
Objective Function: E is just the total length of journey
where the city N+1 is just the city 1.
( ) ( )2 2
1 11
N
i i i ii
E x x y y+ +=
= − + −∑
Monte Carlo Optimization
Annealing Schedule: Require some experiments. First generate some random rearrangements to determine the range of ΔE, then proceed in multiplicative steps each amounting to a 10% decrease in T. Hold a new T for 100Nreconfiguration or 10N successful reconfiguration, whichever comes first. Until efforts to reduce E further becomes discouraging.