Efficient Implementation of Memristor Cellular Nonlinear ...
Applications of Cellular Neural/Nonlinear Networks in Physics
description
Transcript of Applications of Cellular Neural/Nonlinear Networks in Physics
Applications of Cellular Neural/Nonlinear Networks in
Physics
Mária-Magdolna Ercsey-RavaszScientific advisors:
Dr. Prof. Zoltán Néda
Dr. Prof. Tamás Roska
Babes-Bolyai UniversityPéter Pázmány Catholic University
Outline
• CNN computing• A realistic random number generator• Stochastic simulations on CNN computers
• The site-percolation problem• The two-dimensional Ising model
• Optimization of spin-glasses on a space-variant CNN
• Pulse-coupled oscillators communicating with light pulses
The standard CNN modelEach cell has a circuit with:➡ Input voltage: u➡ State voltage: x➡ Output voltage: y
1121
ijijij xxy
ijjiNlkC
kljiNlkC
klijij zulkjiBtylkjiAtxdt
tdx
rr
),(),(),(),(
),;,()(),;,()()(
Template:Template: , A= , B= , z
111011
010010
111011
aaa
aaa
aaa
111011
010010
111011
bbb
bbb
bbb
.
L. O. Chua , L. Yang, IEEE Transactions on
Circuits and Systems 35. No. 10, 1988
The CNN Universal Machine
ACE16K CNN chip: 128*128 cells Bi-i V2
• programmable
• parallel processing
• continuous in time
• continuous (analog) in values
• discrete in space
• Universal (in Turing sense) on integers and on analog array signals
T. Roska, L. O. Chua, IEEE Transactions on Circuits and Systems – II, 40, 1993
CNN computingName Year Size-- 1993 12*12
ACE440 1995 20*22
POS48 1997 48*48
ACE4k 1998 64*64
CACE1k 2001 32*32*2
ACE16k 2002 128*128
XENON 2004 128*96*2
EYE-RIS 2007 176*144
• image processing
• real-time algorithms
• fast and smart camera computer
• robot eyes, bionic eye-glass
• cellular automata models
• partial differential equations
Research goals:• applications in physics
• how should CNN computers be further developed? – from physicist perspectives
Generating realistic random numbers
• Chaotic cellular automaton perturbed
with the natural noise of the chip P’(t)=P(t) xor N(t) N(t): - very few black pixels
- strong correlations but real stochastic fluctuations
Yalcin, et alle., Int.J. Circ. Theor. Appl., 32, 591-607, 2004
),()1,(),1())1,(),1((),(1 jixjixjixjixjixjix tttttt
P(t) P’1(t) XOR P’2(t)
• A good pseudo-random generator
• a good random binary image in t=116 µs• 1 single random value:
• ACE16K 7ns• Pentium 4 at 2.8 GHz (Linux) 33ns
Increasing the size of the chip in the future will
assure even much bigger advantage for
CNN chips
Trend for the simulation time as a function of the chip size
M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of Modern Physics C, Vol. 17, No. 6, p. 909 (2006)
• generating random images with different p density of the black pixels --- using more images with ½ density
--- if p is an n bit number we need n images
p=0.25 p=0.375 p=0.03125P Measured density1/2= 0.5 0.4999529
1/4=0.25 0.254261
1/8=0.125 0.124140
1/16=0.0625 0.061423
1/32=0.03125 0.031561
1/64=0.015625 0.015257
1/128=0.0078125 0.007470
1/256=0.00390625 0.004154
1/4 + 1/8=0.375 0.377712
Correlations:
• in space (first neighbors): 0.05% - 0.4%
• in time (consecutive steps): 0.7% - 0.8 %
M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of Modern Physics C, Vol. 17, No. 6, p. 909 (2006)
Stochastic simulations on CNN computersThe site percolation problem
Used for modeling:- conductivity or mechanical properties of composite materials
- magnetization of dilute magnets at low temperatures
- fluid passing through porous materials
- propagation of diseases
Probability of percolation - density of black pixels
• 2nd order geometrical phase-transition• With CNN: 1 single template detecting percolation
• input: the random image• initial state: the first row• output: the parts connected to the first row
5.05.05.05.045.05.05.05.0
A
000040000
B 3z ACE16k chip
Percolation probability- for each p: 10000 different initial conditions - results agree with the accepted critical value:
pc=0.407
• Time needed:• CNN: t ~ L• digital computers: t ~ L
• For L=128 CNN is slower• if L grows still promises
advantage
2
Trend for the simulation time in function of the chip size
M. Ercsey-Ravasz, T. Roska, Z. Neda, Int. J. of Modern Physics C, Vol. 17, No. 6, p. 909 (2006)
The two-dimensional Ising model
1,
i
jji
i
S
SSJH
kTEepE
pE/0
10
Energy of the system:
On CNN: A parallel Metropolis algorithm is used– Because parallel computing we have to avoid flipping 2 neighbors
simultaneously chessboard mask– Odd (even) step : spins marked with black (white) are updated– Equivalent with a Metropolis algorithm in which spins are
chosen in a well defined order
Metropolis algorithm
- randomly choose a spin and flip it with p probability
• Algorithm scheme for 1 MC step:– Build 3 masks marking: -generate 2 random images
- Spins with 4 similar neighbors (E=8J): M1 ----AND------ P1 with exp(-8J/kT) - Spins with 3 similar neighbors (E=4J): M2 ----AND------ P2 with exp(-4J/kT)- Other spins (E0): M3
– Build the composed mask M=(M1 AND P1) OR (M2 AND P2) OR M3– Use the (inverse) chessboard mask: M’= M AND C in (even) odd steps– Flip the spins marked on M’
T=2 T=2.3 T=2.6 ( J/k=1)Movies obtained with the ACE16K chip
M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys. J. B, Vol. 51, No. 3, p. 407, (2006)
• Initial state: homogeneous• Boundary conditions: fixed • 5000 transition MC steps• Averaging over 10000 MC steps
Results for the Ising model
M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys. J. B, Vol. 51, No. 3, p. 407, (2006)
Magnetization
Specific heat
Susceptibility
Time needed for 1 MC step (128*128 lattice):Time needed for 1 MC step (128*128 lattice): 4.3 ms4.3 ms on the ACE16K chipon the ACE16K chip
2.2 ms2.2 ms on a Pentium on a Pentium 4 at 2.4 GHz4 at 2.4 GHz
M. Ercsey-Ravasz, T. Roska, Z. Neda, Eur. Phys. J. B, Vol. 51, No. 3, p. 407, (2006)
Optimization of spin-glasses on a space-variant CNN
same local minimasame local minima
1 operation 1 operation ↔↔ 1 local minimum 1 local minimum
0
0),;,(,),;,(
1),;,(),,;,(),;,(,1),;,(
z
lkjiBbjijiB
jijiAjilkAlkjiAlkjiA
only simulatedonly simulated
),(
,,,;,
,,,;,ji
jijilkji
lkji yubyylkjiAE
• final state after an operation: yyij ij = = 11
• Lyapunov function (energy) of the CNN:
• monotone decreasing
• final state: local minimum (dE/dt=0)CNN Spin-glassy[-1,1] y=±1
The stochastic optimization algorithm
1 cooling process
Same principles as in simulated annealing:
• noise: random input U
• b ↔ strength of noise
• b slowly decreases
• we choose:
b0=5
Δb=0.05
NP-hard problem
hard for p<0.6
Speed estimation:
• the A templates must be introduced only once for each problem
• we can use the characteristic parameters of the ACE16k chip
1000- 5000 steps / second1000- 5000 steps / secondIndependent of size !Independent of size !
Many applications: error-correcting codes, econophysics, computer science etc.
M. Ercsey-Ravasz, T. Roska, Z. Neda, Physica D: Nonlinear Phenomena, Special issue: “Novel computing paradigms: Quo vadis?, accepted, (2008),
http://dx.doi.org/10.1016/j.physd.2008.03.028
Pulse-coupled oscillators communicating with light pulses
Motivations: - studying a CNN with pulse-coupled oscillators
- communicating with light global coupling
- perspectives: separately programmable oscillators
- first part of the study: collective behavior of identical units
The oscillators:• “electronic fireflies”• simple integrate-and-fire type neurons
• Photoresistor (R,U) + LED
• light R U
• G: threshold
• if U>G LED fires
• not before Tmin
• not after Tmax
Tmin 800 ms
Tmax 2700 ms
Firing 200 ms
Reaction time of the photoresistor 40 ms
Deviations: 2-10 %
Collective behavior
Order parameter: - normalized phase-histogram: smoothing:
n
kiiiik HH
nr
,1
)min()max(1
1
)(ih )(iH
n
kkrn
r1
1
PerspectivesSeparately programmable oscillators:
•Tmin, Tmax, Tflash, light intensity A, threshold G
CNN model using pulse-coupled oscillators:
jilk lk
klklklij ztutyAtx,, ,
)()()(
Benefits: - global coupling
- dynamical inputs
- time-delays
No independent A(i,j;k,l)
- pattern recognition, detecting spatio-temporal events
- studying the role of reaction time Δ
- dynamically changing the parameters
Conclusion• Realistic random numbers• Stochastic simulations on lattice models
- The site-percolation problem- The two-dimensional ising model- many related problems could be also simulated
• Locally variant CNN – very fast stochastic optimization algorithm for spin-glass models
• Further motivating the development of CNN-UM hardwares
• CNN built up by pulse-coupled oscillators– Ineteresting collective behavior – further studies– Communication with light could be useful idea also in hardware
projects
Journal publications
1. M. Ercsey-Ravasz, T. Roska, Z. Néda, “Perspectives for Monte Carlo simulations on the CNN universal machine”, Int. J. of Modern Physics C, Vol. 17, No. 6, pp. 909-923, 2006
2. M. Ercsey-Ravasz, T. Roska, Z. Néda, “Stochastic simulations on the cellular wave computers”, Eur. Phys. J. B, Vol. 51, No. 3, pp. 407-412, 2006
3. M. Ercsey-Ravasz, T. Roska, Z. Néda, “Statistical physics on cellular neural network computers”, Physica D: Nonlinear Phenomena, vol. Special issue: “Novel computing paradigms: Quo Vadis?”, 2008, accepted, http://dx.doi.org/10.1016/j.physd.2008.03.28
International conferences1. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Random number generator and
monte carlo type simulations on the cnn-um,” in Proceedings of the 10th IEEE International Workshop on Cellular Neural Networks and their applications, (Istanbul, Turkey), pp. 47–52, Aug. 2006.
2. M. Ercsey-Ravasz, Z. Sarkozi, Z. Neda, A. Tunyagi, and I. Burda, “Collective behavior of ”electronic fireflies”, SynCoNet 2007: International Symposium on Synchronization in Complex Networks, July 2007.
3. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Statistical physics on cellular neural network computers.” International conference ”Unconventional computing: Quo vadis?”, Mar. 2007.
4. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Spin-glasses on a locally variant cellular neural network.” International Conference on Complex Systems and Networks, July 2007.
5. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “Applications of cellular neural networks in physics.” RHIC Winterschool, Nov. 2005.
6. M. Ercsey-Ravasz, T. Roska, and Z. N´eda, “The cellular neural network universal machine in physics.” International Conference on Computational Methods in Physics, Nov. 2006.
7. M. Ercsey-Ravasz, T. Roska, and Z. Neda, “NP-hard optimization using locally variant CNN,” accepted in the Proceedings of the CNNA2008.
Thank You!