1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore.
-
Upload
virgil-watts -
Category
Documents
-
view
224 -
download
6
Transcript of 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore.
1
Cluster Monte Carlo Cluster Monte Carlo Algorithms Algorithms
Jian-Sheng WangJian-Sheng WangNational University of SingaporeNational University of Singapore
Cluster Monte Carlo Cluster Monte Carlo Algorithms Algorithms
Jian-Sheng WangJian-Sheng WangNational University of SingaporeNational University of Singapore
2
Outlinebull Introduction to Monte Carlo and
statistical mechanical modelsbull Cluster algorithmsbull Replica Monte Carlo
3
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
2
Outlinebull Introduction to Monte Carlo and
statistical mechanical modelsbull Cluster algorithmsbull Replica Monte Carlo
3
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
3
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
1 Introduction to MC 1 Introduction to MC and Statistical and Statistical
Mechanical ModelsMechanical Models
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
4
Stanislaw Ulam (1909-1984)
S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
5
Nicholas Metropolis (1915-1999)
The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
6
The Name of the Game
Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino
Monte-Carlo Monaco
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
7
Use of Monte Carlo Methods
bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling
bull Using random numbers in an essential way
bull Simulation of stochastic processes
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
8
Markov Chain Monte Carlo
bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)
bull Move X by the transition probability W(X -gt Xrsquo)
bull Starting from arbitrary P0(X) we have
Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
9
bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo
bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)
Necessary and sufficient conditions for convergence
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
10
Taking Statisticsbull After equilibration we estimate
1
1( ) ( )P( )d ( )
N
ii
Q X Q X X X Q XN
It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
11
Choice of Transition Matrix W
bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
12
Metropolis Algorithm (1953)
bull Metropolis algorithm takes
W(X-gtXrsquo) = T(X-gtXrsquo) min(1
P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
13
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
14
Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV
(Note the ideal gas law) PV = N kBT
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
15
The Statistical Mechanics of Classical Gas(complex) FluidsSolids
Compute multi-dimensional integral
where potential energy
( 1 1)
1 1 2 2 1 1
( 1 1)
1 1
( )e
e
B
B
E x yk T
N N
E x yk T
N N
Q x y x y dx dy dx dyQ
dx dy dx dy
1( ) ( )N
iji j
E x V d
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
16
The Ising Model
- +
+
+
+
++
+
++
++
+
++
+
+-
---
-- -
- --
- ----
---- The energy of
configuration σ is
E(σ) = - J sumltijgt σi σj
where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1
σ = σ1 σ2 hellip σi hellip
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
17
The Potts Model2
1
3
1
2
3
2
2
2
1
2
2
13
2
2
2
3
32
1
2 2
1 3
3
3 32
2
1
111
1The energy of configuration σ is
E(σ) = - J sumltijgt δ(σiσj)
σi = 12hellipq
1
See F Y Wu Rev Mod Phys 54 (1982) 238 for a review
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
18
Metropolis Algorithm Applied to Ising Model
(Single-Spin Flip)
1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo
is a new configuration with the spin at site I flipped rsquoI=-
3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
19
Boltzmann Distribution
bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution
( ) ( )
( ) ( )
( )e( )
e
E kT
E kT
Z
Z
Z is called partition function
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
20
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
21
Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is
pb (1-p)M-b
b is number of occupied bonds M is total number of bonds
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
22
Fortuin-Kasteleyn Mapping (1969)
( 1)
1 0
(1 )
1
i jij
i j ij ij
c
K
n nn ij
M b Nb
X
Z e
p p
p p q
where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
23
Sweeny Algorithm (1983)
Heat-bath rates
w( -gt1) = p
w( -gt ) = 1 ndash p
w( -gt 1β) = p( (1-p)q +p )
w( -gt β) = (1-p)q( (1-p)q + p )
P(X) ( p(1-p) )b qNc
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
24
Swendsen-Wang Algorithm (1987)
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
An arbitrary Ising configuration according to
( )i j
ij
K
P e
K = J(kT)
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
25
Swendsen-Wang Algorithm
++
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
Put a bond with probability p = 1-e-K if σi = σj
1 0( ) (1 )i j ij ijn n
ij
P n p p
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
26
Swendsen-Wang Algorithm
Erase the spins
1 0
( ) (1 )
(1 )
i j ij ij
c
n nij
Nb M b
P n p p
p p q
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
27
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Assign new spin for each cluster at random Isolated single site is considered a cluster
Go back to P(σn) again
---
- -+
+
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
28
Swendsen-Wang Algorithm
++
++
+
++
+
+
+
+
+
+
++ ++
+
+
+
+
-
-
-
---
-
- --
-- - - -- -
--
- -
Erase bonds to finish one sweep
Go back to P(σ) again
---
- -+
+
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
29
Identifying the Clustersbull Hoshen-Kompelman algorithm
(1976) can be used bull Each sweep takes O(N)
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
30
Measuring Error
bull Let Qt be some quantity of interest at time step t then sample average is
QN = (1N) sumt Qt
bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN
2 = ltQN2gt-ltQNgt2
lthellipgt standards for average over the exact distribution
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
31
Estimating Variance
22
1 1
int
1
var( ) ( ) 1
var( )
N
N t s t st s
N
t N
Q Q Q QN
tQf t
N N
QN
H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
32
Error Formulabull The above derivation gives the well-
known error estimate in Monte Carlo as
where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt
intvar( ) 1Error N
QN N
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
33
Time-Dependent Correlation Function and
Integrated Correlation Time
bull We define
and
22( ) s s t s s t
s s
Q Q Q Qf t
Q Q
int0 1 2 1
( ) 1 2 ( )t t
f t f t
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
34
Critical Slowing Down
Tc T
The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
35
Much Reduced Critical Slowing Down
Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model
From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86
Lz
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
36
Comparison of integrated autocorrelation times at Tc for 2D Ising model
J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
37
Wolff Single-Cluster Algorithm
void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
38
Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
39
Slowing Down at First-Order Phase Transition
bull At first-order phase transition the longest time scale is controlled by the interface barrier
where β=1(kBT) σ is interface free energy d is dimension L is linear size
12 dLe
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
40
Spin Glass Model+
+
++
+
+
+
+
+
+ +
+++
+
+
+
+
++
+
+ ++
-
-
- -- -
-- -
- -- -
-- - - -- - -
- - - -
A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)
( ) ij i jij
E J
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
41
Replica Monte Carlobull A collection of M systems at
different temperatures is simulated in parallel allowing exchange of information among the systems
β1 β2 β3 βM
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
42
Moves between Replicas
bull Consider two neighboring systems σ1 and σ2 the joint distribution is
P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]
bull Any valid Monte Carlo move should preserve this distribution
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
43
Pair Hamiltonian in Replica Monte Carlo
bull We define i=σi1σi
2 then Hpair can be rewritten as
1 1pair
1 2
where
( )
ij i jij
ij i j ij
H K
K J
The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
44
Cluster Flip in Replica Monte Carlo
= +1 = -
1
Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is
Hcl = - Σ kbc sbsc
Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij
bc
Metropolis algorithm is used to flip the clusters ie σi
1 -gt -σi1 σi
2 -gt -σi2 fixing
for all i in a given cluster
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
45
Comparing Correlation Times
Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice
From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607
Replica MC
Single spin flip
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
46
2D Spin Glass Susceptibility
2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps
From J S Wang and R H Swendsen PRB 38 (1988) 4840
K511 was concluded
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
47
Heat Capacity at Low T
c T 2exp(-2JT)
This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -
2
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
48
Monte Carlo Renormalization Group
YH defined by
with RG iterations for difference sizes in 2D
From J S Wang and R H Swendsen PRB 37 (1988) 7745
H
( ) ( 1)
( ) ( )
( ) ( )
[ ]2
[ ]
n nJy
n nJ
n ni
i
q q
q q
q
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
49
MCRG in 3D3D result of YH
MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems
50
Conclusionbull Monte Carlo methods have broad
applicationsbull Cluster algorithms eliminate the
difficulty of critical slowing downbull Replica Monte Carlo works on
frustrated and disordered systems