Parallel Computational Complexity and Statistical...
Transcript of Parallel Computational Complexity and Statistical...
Parallel ComputationalComplexity and Statistical
Physics
Jon MachtaUniversity of Massachusetts Amherst
Supported by the National Science Foundation
Outline
• Parallel computing and computational complexity
• Complexity of models in statistical physical
• Physical complexity and computational complexity
Collaborators
• Ray Greenlaw, Armstrong Atlantic University• Cris Moore, University of New Mexico, SFI
• Ken Moriarty, UMass• Xuenan Li, UMass
• Dan Tillberg, UMass• Ben Machta, Brown University
Computational Complexity•How do computational resources scale with the size ofthe problem?
TimeHardware
•Equivalent models of computation.Turing machineParallel random access machineBoolean circuit familyFormal logic
Boolean Circuit Family
NOR NOR
NOR NOR NOR
NOR NOR
INPUT INPUT INPUT
OUTPUT OUTPUT
width
depth
•Gates evaluated one levelat a time from input tooutput with no feedback.•One hardwired circuit foreach problem size.•Primary resources
Depth=number of levels, DcWidth=maximum numberof gates in a levelWork=total number of gates
Parallel ComputingAdding n numbers can be carried out in O(log n) steps
using O(n) processors.
+X1 X2
+X3 X4
+X5 X6
+X7 X8
+ +
+
ΣXi
log n
Connected components of a graph can be found inO(log2n) steps using n2 processors.
Parallel Random AccessMachine
1 2 3 m
Controller
Global Memory
Processors
PRAM•Each processor runs thesame program but has adistinct label
•Each processorcommunicates with anymemory cell in a singletime step.
•Primary resources:Parallel time ~ depth
Number of processors~width
Complexity Classes andP-completeness
•P is the class of feasible problems: solvable withpolynomial work.•NC is the class of problems efficiently solved inparallel (polylog depth and polynomial work, NC ⊆ P).
•Are there feasible problems that cannot be solvedefficiently in parallel (P≠NC)?
•P-complete problems are the hardest problems in P tosolve in parallel. It is believed they are inherentlysequential: not solvable in polylog depth.•The Circuit Value Problem is P-complete.
NOR NOR
NOR NOR NOR
NOR
TRUE FALSE TRUE
?
Statistical Physics
The study of the emergent properties of manyparticle system using probabilistic methods.Objects of study are statistical ensembles ofsystem states or histories:
•Equilibrium states--Gibbs distribution•Non-equilibrium states--stochastic dynamics
Computational statistical physics: sample theseensembles.
Sampling Complexity
•Monte Carlo simulationsconvert random bits intodescriptions of a typicalsystem states.•What is the depth of theshallowest feasible circuit(running time of the fastestPRAM program) thatgenerates typical states?
NOR NOR
NOR NOR NOR
NOR NOR
Typical System State
depthRandom Bits
Depth is a property of systems in statistical physics
Diffusion Limited Aggregation
•Particles added one at a timewith sticking probabilitiesgiven by the solution ofLaplace’s equation.•Self-organized fractal object
df=1.715… (2D)•Physical systems:
Fluid flow in porous mediaElectrodepositionBacterial colonies
Witten and Sander, PRL 47, 1400 (1981)
Random Walk Dynamics for DLA
#1#3#2 #1 #3
#2
Parallel dynamics ignoresinterference between 1 and 3
Sequential dynamics
The Problem with Parallelizing DLA
Depth of DLATheorem: Determining the shape of an aggregate from therandom walks of the constituent particles is a P-completeproblem.Proof sketch: Reduce the Circuit Value Problem to DLA dynamics.
Caveats:1. P≠NC not proven2. Average case may be easier than worst case3. Alternative dynamics may be easier than random walk dynamics
a b c
input 1 input 2
power
output
d
132
tentative cluster, step N tentative cluster, step N+1
1. Start with seed particle at the origin and N walk trajectories2. In parallel move all particles along their trajectories to tentative sticking
points on tentative cluster, which is initially the seed particle at the origin.3. New tentative cluster obtained by removing all particles that interfere
with earlier particles.4. Continue until all particles are correctly placed.
Parallel Algorithm for DLAD. Tillberg and JM, PRE 69, 051403 (2004)
2 1
Efficiency of the Algorithm•DLA is a tree whosestructural depth , Ds scalesas the radius of the cluster.•The running time, T of thealgorithm is asymptoticallyproportional to the structuraldepth.
Ds/T
T
Internal DLAParticles start at the origin, random walk and stick where theyfirst leaves the cluster.
•Shape approaches a circle with logarithmic fluctuations.•P-completeness proof fails.
Parallel Algorithm for IDLA1. Start with seed particle at the origin and N walk trajectories2. Place particles at expected positions along their trajectories.3. Iteratively move particles until holes and multiple occupancies are
eliminated
Average parallel timepolylogarthmic orpossibly a small powerin N.
Cluster of 2500 particles made in 6 parallel steps.
1
2 4
3 5
6
C. Moore and JM, J. Stat. Phys. 99, 661 (2000)
Hierarchy of Depth vs Size
Mandelbrot percolation DLA
constant polylog polynomial
T>Tc Ising T=Tc IsingEden growth
Invasion percolation
Eden growthInvasion percolationScale free networksBallistic depositionBak-Sneppen model
Internal DLA
What is Physical Complexity?
• “I shall not today attempt further to define the kindsof material I understand to be embraced with thatshorthand description. ... But I know it when I see it.”– Justice Potter Stewart on pornography
< ≤ <<
History and Complexity
• The emergence of a complex system fromsimple initial conditions requires a long history.
• History can be quantified in terms of thecomputational complexity of simulating statesof the system.
–Charles Bennett
Depth is the appropriate measure to quantify history.
Properties of Depth• Defined for any system in the framework of
statistical physics.• For a system of nearly independent subsystems
given by the maximum over the subsystems.• Greatest at the boundary between order and
disorder.• Systems that solve (P-hard) computational
problems are deep.
…and physical complexity
+ ≈
Depth of a system of nearly independentsubsystems given by the maximum overthe subsystems.
≈Depth is intensive (independent ofsize) for homogeneous systemswith short range correlations.
Follows immediately from parallelism
Depth is greatest at the boundary betweenorder and disorder.
In equilibrium systems with continuousphase transitions, such as the Ising model,depth is greatest at the critical point,separating order and disorder (criticalslowing down).
<< <<
Conclusions
Parallel computational complexity theory providesinteresting perspectives on model systems in statisticalphysics.Depth, defined as the minimum number of parallel stepsneeded to simulate a system, is a prerequisite for physicalcomplexity and has many of its properties.