8/11/2019 New Optimization Algorithms in Physics_1
1/10
New Optimization
Algorithms in Physics
Edited byAlexander K. Hartmann and Heiko Rieger
New Optimization Algorithms in Physics.Edited by Alexander K. Hartmann, Heiko Rieger
Copyright c 2004 Wiley-VCH Verlag GmbH & Co. KGaA
ISBN: 3-527-40406-6
8/11/2019 New Optimization Algorithms in Physics_1
2/10
New Optimization
Algorithms in Physics
Edited byAlexander K. Hartmann and Heiko Rieger
8/11/2019 New Optimization Algorithms in Physics_1
3/10
Editors
Alexander K.Hartmann
Universitt Gttingen, Germany
Heiko Rieger
Universitt des Saarlandes, Germany
Cover Picture
Short artificial peptide (U. H. E. Hansmann);
Dense configuration of polydisperse hard discs (W.
Krauth); Cut Polytope for the complete graph withN=3 (F. Liers et al); Factor graph for 3-SAT (R.
Zecchina)
This book was carefully produced.Nevertheless,
editors,authors and publisher do not warrant the
information contained therein to be free of errors.Readers are advised to keep in mind that state-
ments,data, illustrations, procedural details orother items may inadvertently be inaccurate.
Library of Congress Card No.: applied for
British Library Cataloging-in-Publication Data:
A catalogue record for this book is available fromthe British Library
Bibliographic information published by
Die Deutsche Bibliothek
Die Deutsche Bibliothek lists this publication inthe Deutsche Nationalbibliografie; detailed bibli-
ographic data is available in the Internet at.
2004 WILEY-VCH Verlag GmbH & Co. KGaA,
Weinheim
All rights reserved (including those of translationinto other languages). No part of this book may
be reproduced in any form nor transmitted or
translated into machine language without writtenpermission from the publishers. Registered
names, trademarks, etc. used in this book, evenwhen not specifically marked as such, are not to
be considered unprotected by law.
Printed in the Federal Republic of Germany
Printed on acid-free paper
Composition Uwe Krieg,Berlin
Printing Strauss GmbH, MrlenbachBookbinding Litges & Dopf Buchbinderei
GmbH, HeppenheimISBN 3-527-40406-6
8/11/2019 New Optimization Algorithms in Physics_1
4/10
Contents
List of Contributors XI
1 Introduction
(A.K. Hartmann and H. Rieger
) 1
Part I Applications in Physics 5
2 Cluster Monte Carlo Algorithms
(W. Krauth) 7
2.1 Detailed Balance and a priori Probabilities . . . . . . . . . . . . . . . . . . . 7
2.2 The Wolff Cluster Algorithm for the Ising Model . . . . . . . . . . . . . . . 10
2.3 Cluster Algorithm for Hard Spheres and Related Systems . . . . . . . . . . . 12
2.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.1 Phase Separation in Binary Mixtures . . . . . . . . . . . . . . . . . . 16
2.4.2 Polydisperse Mixtures . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.3 Monomer-Dimer Problem . . . . . . . . . . . . . . . . . . . . . . . 19
2.5 Limitations and Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3 Probing Spin Glasses with Heuristic Optimization Algorithms
(O.C. Martin) 23
3.1 Spin Glasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1.1 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1.2 The Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1.3 Models of Spin Glasses . . . . . . . . . . . . . . . . . . . . . . . . . 243.1.4 Some Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2 Some Heuristic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.1 General Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.2 Variable Depth Search . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2.3 Genetic Renormalization Algorithm . . . . . . . . . . . . . . . . . . 36
3.3 A Survey of Physics Results . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.3.1 Convergence of the Ground-state Energy Density . . . . . . . . . . . 41
3.3.2 Domain Walls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.3.3 Clustering of Ground States . . . . . . . . . . . . . . . . . . . . . . 42
3.3.4 Low-energy Excitations . . . . . . . . . . . . . . . . . . . . . . . . 42
8/11/2019 New Optimization Algorithms in Physics_1
5/10
VI Contents
3.3.5 Phase Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4 Computing Exact Ground States of Hard Ising Spin Glass Problems byBranch-and-cut
(F. Liers, M. Jnger, G. Reinelt, and G. Rinaldi) 47
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2 Ground States and Maximum Cuts . . . . . . . . . . . . . . . . . . . . . . . 48
4.3 A General Scheme for Solving Hard Max-cut Problems . . . . . . . . . . . . 51
4.4 Linear Programming Relaxations of Max-cut . . . . . . . . . . . . . . . . . 55
4.5 Branch-and-cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.6 Results of Exact Ground-state Computations . . . . . . . . . . . . . . . . . . 62
4.7 Advantages of Branch-and-cut . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.8 Challenges for the Years to Come . . . . . . . . . . . . . . . . . . . . . . . 66References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5 Counting States and Counting Operations
(A. Alan Middleton) 71
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.2 Physical Questions about Ground States . . . . . . . . . . . . . . . . . . . . 72
5.2.1 Homogeneous Models . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.2.2 Magnets with Frozen Disorder . . . . . . . . . . . . . . . . . . . . . 73
5.3 Finding Low-energy Configurations . . . . . . . . . . . . . . . . . . . . . . 75
5.3.1 Physically Motivated Approaches . . . . . . . . . . . . . . . . . . . 755.3.2 Combinatorial Optimization . . . . . . . . . . . . . . . . . . . . . . 76
5.3.3 Ground-state Algorithm for the RFIM . . . . . . . . . . . . . . . . . 78
5.4 The Energy Landscape: Degeneracy and Barriers . . . . . . . . . . . . . . . 80
5.5 Counting States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.5.1 Ground-state Configuration Degeneracy . . . . . . . . . . . . . . . . 83
5.5.2 Thermodynamic State . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.5.3 Numerical Studies of Zero-temperature States . . . . . . . . . . . . . 86
5.6 Running Times for Optimization Algorithms . . . . . . . . . . . . . . . . . . 91
5.6.1 Running Times and Evolution of the Heights . . . . . . . . . . . . . 92
5.6.2 Heuristic Derivation of Running Times . . . . . . . . . . . . . . . . 945.7 Further Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6 Computing the Potts Free Energy and Submodular Functions
(J.-C. Angls dAuriac) 101
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.2 The Potts Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.2.1 Definition of the Potts Model . . . . . . . . . . . . . . . . . . . . . . 102
6.2.2 Some Results for Non-random Models . . . . . . . . . . . . . . . . . 103
6.2.3 The Ferromagnetic Random Bond Potts Model . . . . . . . . . . . . 103
8/11/2019 New Optimization Algorithms in Physics_1
6/10
Contents VII
6.2.4 High Temperature Development . . . . . . . . . . . . . . . . . . . . 103
6.2.5 Limit of an Infinite Number of States . . . . . . . . . . . . . . . . . 104
6.3 Basics on the Minimization of Submodular Functions . . . . . . . . . . . . . 105
6.3.1 Definition of Submodular Functions . . . . . . . . . . . . . . . . . . 1056.3.2 A Simple Characterization . . . . . . . . . . . . . . . . . . . . . . . 105
6.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.3.4 Minimization of Submodular Function . . . . . . . . . . . . . . . . . 106
6.4 Free Energy of the Potts Model in the Infinite q-Limit . . . . . . . . . . . . . 107
6.4.1 The Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.4.2 The Auxiliary Problem . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.4.3 The Max-flow Problem: the Goldberg and Tarjan Algorithm . . . . . 111
6.4.4 About the Structure of the Optimal Sets . . . . . . . . . . . . . . . . 111
6.5 Implementation and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.5.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1126.5.2 Example of Application . . . . . . . . . . . . . . . . . . . . . . . . 114
6.5.3 Evaluation of the CPU Time . . . . . . . . . . . . . . . . . . . . . . 114
6.5.4 Memory Requirement . . . . . . . . . . . . . . . . . . . . . . . . . 114
6.5.5 Various Possible Improvements . . . . . . . . . . . . . . . . . . . . 115
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Part II Phase Transitions in Combinatorial Optimization Problems 119
7 The Random 3-satisfiability Problem: From the Phase Transition to the Efficient
Generation of Hard, but Satisfiable Problem Instances
(M. Weigt) 121
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.2 Random 3-SAT and the SAT/UNSAT Transition . . . . . . . . . . . . . . . . 122
7.2.1 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.2.2 Using Statistical Mechanics . . . . . . . . . . . . . . . . . . . . . . 124
7.3 Satisfiable Random 3-SAT Instances . . . . . . . . . . . . . . . . . . . . . . 127
7.3.1 The Naive Generator . . . . . . . . . . . . . . . . . . . . . . . . . . 129
7.3.2 Unbiased Generators . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
8 Analysis of Backtracking Procedures for Random Decision Problems
(S. Cocco, L. Ein-Dor, and R. Monasson) 139
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.2 Phase Diagram, Search Trajectories and the Easy SAT Phase . . . . . . . . . 143
8.2.1 Overview of Concepts Useful to DPLL Analysis . . . . . . . . . . . 144
8.2.2 Clause Populations: Flows, Averages and Fluctuations . . . . . . . . 145
8.2.3 Average-case Analysis in the Absence of Backtracking . . . . . . . . 147
8.2.4 Occurrence of Contradictions and Polynomial SAT Phase . . . . . . . 150
8/11/2019 New Optimization Algorithms in Physics_1
7/10
VIII Contents
8.3 Analysis of the Search Tree Growth in the UNSAT Phase . . . . . . . . . . . 153
8.3.1 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . 153
8.3.2 Parallel Growth Process and Markovian Evolution Matrix . . . . . . 155
8.3.3 Generating Function and Large-size Scaling . . . . . . . . . . . . . . 158
8.3.4 Interpretation in Terms of Growth Process . . . . . . . . . . . . . . . 1618.4 Hard SAT Phase: Average Case and Fluctuations . . . . . . . . . . . . . . . 164
8.4.1 Mixed Branch and Tree Trajectories . . . . . . . . . . . . . . . . . . 164
8.4.2 Distribution of Running Times . . . . . . . . . . . . . . . . . . . . . 165
8.4.3 Large Deviation Analysis of the First Branch in the Tree . . . . . . . 167
8.5 The Random Graph Coloring Problem . . . . . . . . . . . . . . . . . . . . . 171
8.5.1 Description of DPLL Algorithm for Coloring . . . . . . . . . . . . . 171
8.5.2 Coloring in the Absence of Backtracking . . . . . . . . . . . . . . . 172
8.5.3 Coloring in the Presence of Massive Backtracking . . . . . . . . . . 173
8.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
9 New Iterative Algorithms for Hard Combinatorial Problems
(R. Zecchina) 183
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
9.2 Combinatorial Decision Problems, K-SAT and the Factor Graph Representation185
9.2.1 Random K-SAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
9.3 Growth Process Algorithm: Probabilities, Messages and Their Statistics . . . 190
9.4 Traditional Message-passing Algorithm: Belief Propagation as Simple Cavity
Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
9.5 Survey Propagation Equations . . . . . . . . . . . . . . . . . . . . . . . . . 1949.6 Decimating Variables According to Their Statistical Bias . . . . . . . . . . . 195
9.7 Conclusions and Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . 197
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Part III New Heuristics and Interdisciplinary Applications 203
10 Hysteretic Optimization
(K.F. Pl) 205
10.1 Hysteretic Optimization for Ising Spin Glasses . . . . . . . . . . . . . . . . . 206
10.2 Generalization to Other Optimization Problems . . . . . . . . . . . . . . . . 21410.3 Application to the Traveling Salesman Problem . . . . . . . . . . . . . . . . 221
10.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
11 Extremal Optimization
(S. Boettcher) 227
11.1 Emerging Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
11.2 Extremal Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
11.2.1 Basic Notions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
11.2.2 EO Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
8/11/2019 New Optimization Algorithms in Physics_1
8/10
Contents IX
11.2.3 Extremal Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
11.2.4 Rank Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
11.2.5 Defining Fitness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
11.2.6 Distinguishing EO from other Heuristics . . . . . . . . . . . . . . . 235
11.2.7 Implementing EO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
11.3 Numerical Results for EO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
11.3.1 Early Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
11.3.2 Applications of EO by Others . . . . . . . . . . . . . . . . . . . . . 242
11.3.3 Large-scale Simulations of Spin Glasses . . . . . . . . . . . . . . . . 243
11.4 Theoretical Investigations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
12 Sequence Alignments
(A.K. Hartmann) 253
12.1 Molecular Biology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
12.2 Alignments and Alignment Algorithms . . . . . . . . . . . . . . . . . . . . . 259
12.3 Low-probability Tail of Alignment Scores . . . . . . . . . . . . . . . . . . . 266
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
13 Protein Folding in Silico the Quest for Better Algorithms
(U.H.E. Hansmann) 275
13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
13.2 Energy Landscape Paving . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
13.3 Beyond Global Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 280
13.3.1 Parallel Tempering . . . . . . . . . . . . . . . . . . . . . . . . . . . 28013.3.2 Multicanonical Sampling and Other Generalized-ensemble Techniques 283
13.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
13.4.1 Helix Formation and Folding . . . . . . . . . . . . . . . . . . . . . . 288
13.4.2 Structure Predictions of Small Proteins . . . . . . . . . . . . . . . . 290
13.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
Index 297
8/11/2019 New Optimization Algorithms in Physics_1
9/10
List of Contributors
Jean-Christian Angls dAuriac, Ch. 6
Centre de Recherches sur les Tres Basses Tem-
peratures
BP 166, F-38042 Grenoble, France
e-mail: [email protected]
Stefan Boettcher, Ch. 11
Stefan Boettcher
Physics Department
Emory University
Atlanta, GA 30322; USA
e-mail: [email protected]
www.physics.emory.edu/faculty/boettcher
Simona Cocco, Ch. 8
Laboratoire de Dynamique des Fluides Com-
plexes
3 rue de lUniversit
67000 Strasbourg, France
e-mail: [email protected]
Liat Ein-Dor, Ch. 8
Department of Physics of Complex Systems
Wezmann Institute
Rehovot 76100, Israel
and
Laboratoire de Physique Thorique de lENS
24 rue Lhomond
75005 Paris, France
Ulrich H. E. Hansmann, Ch. 13
Department of Physics
Michigan Technological University
Houghton
MI 49931-1295, USA
e-mail: [email protected]
Alexander K. Hartmann, Ch. 1, 12
Institut fr Theoretische Physik
Universitt Gttingen
Tammanstrae 1
D-37077 Gttingen, Germany
e-mail:
Michael Jnger, Ch. 4
Institut fr Informatik
Universitt zu Kln
Pohligstrae 1
D-50969 Kln, Germany
e-mail: [email protected]
Werner Krauth, Ch. 2
CNRS-Laboratoire de Physique Statistique
Ecole Normale Suprieure24, rue Lhomond
F-75231 Paris Cedex 05, France
e-mail: [email protected]
Frauke Liers, Ch. 4
Institut fr Informatik
Universitt zu Kln
Pohligstrae 1
D-50969 Kln, Germany
e-mail: [email protected]
New Optimization Algorithms in Physics.Edited by Alexander K. Hartmann, Heiko Rieger
Copyright c 2004 Wiley-VCH Verlag GmbH & Co. KGaA
ISBN: 3-527-40406-6
8/11/2019 New Optimization Algorithms in Physics_1
10/10
XII List of Contributors
Olivier C. Martin, Ch. 3
LPTMS, Universit Paris-Sud
Orsay Cedex 91405, France
e-mail: [email protected]
A. Alan Middleton, Ch. 5
Department of Physics
Syracuse University
Syracuse, NY, USA
e-mail: [email protected]
Remi Monasson, Ch. 8Laboratoire de Physique Thorique de lENS
24 rue Lhomond
75005 Paris, France
e-mail: [email protected]
and
Laboratoire de Physique Thorique
3 rue de lUniversit67000 Strasbourg, France
Kroly F. Pl, Ch. 10
Department of Theoretical Physics
Institute of Nuclear Research of the
Hungarian Academy of Sciences
Bem tr 18/c
H-4026 Debrecen, Hungary
e-mail: [email protected]
Gerhard Reinelt, Ch. 4
Institut fr Informatik
Ruprecht-Karls-Universitt Heidelberg
Im Neuenheimer Feld 368
D-69120 Heidelberg, Germany
e-mail:
Heiko Rieger, Ch. 1
Theoretische Physik
Universitt des Saarlandes
D-66041 Saarbrcken, Germany
e-mail: [email protected]
Giovanni Rinaldi, Ch. 4
Istituto di Analisi dei Sistemi ed Informatica
Antonio Ruberti - CNR
Viale Manzoni, 30
00185 Roma, Italy
e-mail: [email protected]
Martin Weigt, Ch. 7
Institut fr Theoretische Physik
Universitt GttingenTammanstrae 1
D-37077 Gttingen, Germany
e-mail: [email protected]
Riccardo Zecchina, Ch. 9
International Centre for Theoretical Physics
(ICTP)
Strada Costiera, 11 P.O.Box 586
I-34100 Trieste, Italy
e-mail: [email protected]
Top Related