Dynamics on Asynchronous Networks
Transcript of Dynamics on Asynchronous Networks
![Page 1: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/1.jpg)
Dynamics on Asynchronous Networks
Mike Field
Chris Bick & Anushaya Mohapatra
Rice University
Research supported in part by NSF Grants DMS-0806321 & DMS-1265253
September, 2013.Asynchronous Networks – p. 1/60
![Page 2: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/2.jpg)
Topics
Asynchronous Networks – p. 2/60
![Page 3: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/3.jpg)
Topics
• Classical dynamics, synchronous networks.
Asynchronous Networks – p. 2/60
![Page 4: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/4.jpg)
Topics
• Classical dynamics, synchronous networks.
• Contemporary problems; asynchronous networks.
Asynchronous Networks – p. 2/60
![Page 5: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/5.jpg)
Topics
• Classical dynamics, synchronous networks.
• Contemporary problems; asynchronous networks.
Caution: SynchronousandAsynchronousmaynot mean what you think...
Asynchronous Networks – p. 2/60
![Page 6: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/6.jpg)
Topics
• Classical dynamics, synchronous networks.
• Contemporary problems; asynchronous networks.
Caution: SynchronousandAsynchronousmaynot mean what you think...
• Examples of asynchronous networks.
Asynchronous Networks – p. 2/60
![Page 7: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/7.jpg)
Topics
• Classical dynamics, synchronous networks.
• Contemporary problems; asynchronous networks.
Caution: SynchronousandAsynchronousmaynot mean what you think...
• Examples of asynchronous networks.
• Dynamics on asynchronous networks and someoutstanding challenges.
Asynchronous Networks – p. 2/60
![Page 8: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/8.jpg)
Classical dynamics
x′ = f(x)
xn+1 = f(xn), n ≥ 0.
Typically, f is assumed to bereal analyticor even apolynomial.
Examples:
Celestial mechanics (from 1687).
Nonlinear oscillators (1920, Van der Pol)
Chaotic dynamics (1963, Lorenz)
Asynchronous Networks – p. 3/60
![Page 9: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/9.jpg)
NetworksIn dynamics it is often natural to group variablestogether leading to the concept of a network.
Example: N-body problem of celestial mechanics
P4
P1
P3P2
Network graph for the4-body problemAsynchronous Networks – p. 4/60
![Page 10: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/10.jpg)
EquationsIn caseN = 4:
X′1 = F1(X1;X2 −X1, · · · ,X4 −X1)
· · · = · · ·X
′4 = F4(X4;X1 −X4, · · · ,X3 −X4),
whereXi = (x1, x2, x3, v1, v2, v3) and theFi are realanalytic.
Asynchronous Networks – p. 5/60
![Page 11: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/11.jpg)
EquationsIn caseN = 4:
X′1 = F1(X1;X2 −X1, · · · ,X4 −X1)
· · · = · · ·X
′4 = F4(X4;X1 −X4, · · · ,X3 −X4),
whereXi = (x1, x2, x3, v1, v2, v3) and theFi are realanalytic.
• In caseN = 1, can reduce to a constant – zerodimensional (relative to a rotating coordinateframe).
Asynchronous Networks – p. 5/60
![Page 12: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/12.jpg)
EquationsIn caseN = 4:
X′1 = F1(X1;X2 −X1, · · · ,X4 −X1)
· · · = · · ·X
′4 = F4(X4;X1 −X4, · · · ,X3 −X4),
whereXi = (x1, x2, x3, v1, v2, v3) and theFi are realanalytic.
• In caseN = 1, can reduce to a constant – zerodimensional (relative to a rotating coordinateframe).
• Note implications of analyticity: coherence, nodeinter-dependence, no stops.
Asynchronous Networks – p. 5/60
![Page 13: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/13.jpg)
Phase oscillator networksNetworks ofN weakly coupled nonlinear oscillatorscan sometimes be modelled by networks ofphaseoscillators(Kuramoto, 1984):
θ′i = ωi +1
N
∑
j 6=i
gij(θj − θi), i = 1, · · · , N.
Hereθi ∈ T = [0, 1]/0=1 and thegij aretrigonometric polynomials.A popular choice is to assumegij = G, all i, j and
G(θ) = α sin(2πθ) + β sin(4πθ)
Also allow sin(2πθ + γ) etc.(All-to-all coupled, symmetric network.)
Asynchronous Networks – p. 6/60
![Page 14: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/14.jpg)
But why networks?
Asynchronous Networks – p. 7/60
![Page 15: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/15.jpg)
But why networks?
• Understanding network dynamics in terms ofnetwork topology?
Asynchronous Networks – p. 7/60
![Page 16: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/16.jpg)
But why networks?
• Understanding network dynamics in terms ofnetwork topology?
• Maybe for small networks ... 4 or 5 identicalnodes and perhaps in some statistical sense forlarge networks.
Asynchronous Networks – p. 7/60
![Page 17: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/17.jpg)
But why networks?
• Understanding network dynamics in terms ofnetwork topology?
• Maybe for small networks ... 4 or 5 identicalnodes and perhaps in some statistical sense forlarge networks.
• Reductionism. Understand dynamics ofindividualnodes and then infer properties aboutdynamics of the complete network in terms of thenode dynamics.
Asynchronous Networks – p. 7/60
![Page 18: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/18.jpg)
But why networks?
• Understanding network dynamics in terms ofnetwork topology?
• Maybe for small networks ... 4 or 5 identicalnodes and perhaps in some statistical sense forlarge networks.
• Reductionism. Understand dynamics ofindividualnodes and then infer properties aboutdynamics of the complete network in terms of thenode dynamics.
• Appropriate (and well-known) forlinearnetworks. What about thenonlinearcase?
Asynchronous Networks – p. 7/60
![Page 19: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/19.jpg)
ReductionismN -body problem. Not helpful. Individual nodes haveno dynamics!
Asynchronous Networks – p. 8/60
![Page 20: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/20.jpg)
ReductionismN -body problem. Not helpful. Individual nodes haveno dynamics!
Phase oscillator systems? Obvious problem – alsooccurring withN -body problem – is that nodes in thenetwork do noteverevolve independently of the othernodes (analyticity again). However, there is one casewhen we can use reductionist logic:
Asynchronous Networks – p. 8/60
![Page 21: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/21.jpg)
ReductionismN -body problem. Not helpful. Individual nodes haveno dynamics!
Phase oscillator systems? Obvious problem – alsooccurring withN -body problem – is that nodes in thenetwork do noteverevolve independently of the othernodes (analyticity again). However, there is one casewhen we can use reductionist logic:
Assume nodes synchronized:θi = θj, ωi = ω, all i, j.We have a solutionθi(t) = θ0 + tω. That is, we canreplace the network by a single phase oscillator(compare the1-body problem analysis).
Asynchronous Networks – p. 8/60
![Page 22: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/22.jpg)
Properties of classical networksFixed connection structure– can assume connectedgraph (else network splits into independent connectedcomponents).
=⇒ nodes never evolve independently of one another.
Asynchronous Networks – p. 9/60
![Page 23: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/23.jpg)
Properties of classical networksFixed connection structure– can assume connectedgraph (else network splits into independent connectedcomponents).
=⇒ nodes never evolve independently of one another.
Nodes never stop and then later restart (consequenceof analyticity).
Asynchronous Networks – p. 9/60
![Page 24: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/24.jpg)
Properties of classical networksFixed connection structure– can assume connectedgraph (else network splits into independent connectedcomponents).
=⇒ nodes never evolve independently of one another.
Nodes never stop and then later restart (consequenceof analyticity).
One set of dynamical equations – no switchingbetween equations.
Asynchronous Networks – p. 9/60
![Page 25: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/25.jpg)
Properties of classical networksFixed connection structure– can assume connectedgraph (else network splits into independent connectedcomponents).
=⇒ nodes never evolve independently of one another.
Nodes never stop and then later restart (consequenceof analyticity).
One set of dynamical equations – no switchingbetween equations.
Global clock– all nodes run on same time(simultaneous evolution of nodes).
Asynchronous Networks – p. 9/60
![Page 26: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/26.jpg)
Global & Local time; Synchronous
dxdt
= f(x,y) dydt
= g(x,y)
dxds
= af(x,y) dydt
= g(x,y)
t = as, a > 0.
Changing time on one nodeEven for linear systems, changing time on a singlenode will usually qualitatively change dynamics.
Asynchronous Networks – p. 10/60
![Page 27: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/27.jpg)
Global & Local time; Synchronous
dxdt
= f(x,y) dydt
= g(x,y)
dxds
= af(x,y) dydt
= g(x,y)
t = as, a > 0.
Changing time on one nodeEven for linear systems, changing time on a singlenode will usually qualitatively change dynamics.
We call networks satisfying the properties listedpreviouslysynchronous networks. This should not beconfused with synchronized dynamics – ourterminology comes from computer science.
Asynchronous Networks – p. 10/60
![Page 28: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/28.jpg)
Asynchronous networksIn anasynchronous networkwe allow
• Variable connectivity – key property:=⇒dependency relationships between nodes vary.
Asynchronous Networks – p. 11/60
![Page 29: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/29.jpg)
Asynchronous networksIn anasynchronous networkwe allow
• Variable connectivity – key property:=⇒dependency relationships between nodes vary.
• Switching between dynamical equations.
Asynchronous Networks – p. 11/60
![Page 30: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/30.jpg)
Asynchronous networksIn anasynchronous networkwe allow
• Variable connectivity – key property:=⇒dependency relationships between nodes vary.
• Switching between dynamical equations.
• Local clocks - no natural global clock (dynamicsnot synchronized to global clock).
Asynchronous Networks – p. 11/60
![Page 31: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/31.jpg)
Asynchronous networksIn anasynchronous networkwe allow
• Variable connectivity – key property:=⇒dependency relationships between nodes vary.
• Switching between dynamical equations.
• Local clocks - no natural global clock (dynamicsnot synchronized to global clock).
• Nodes to stop and later restart.
Asynchronous Networks – p. 11/60
![Page 32: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/32.jpg)
Asynchronous networksIn anasynchronous networkwe allow
• Variable connectivity – key property:=⇒dependency relationships between nodes vary.
• Switching between dynamical equations.
• Local clocks - no natural global clock (dynamicsnot synchronized to global clock).
• Nodes to stop and later restart.
All of these characteristics are typical of networksencountered in modern technology (eg distributednetworks) and science (especially biology andneuroscience). One might argue that synchronousnetworks areatypical in the 21st century.
Asynchronous Networks – p. 11/60
![Page 33: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/33.jpg)
Node clocks
a before b,c,e .. ?d?b before e .. ?c,d?c ..?b,e?d before c,e .. ?a,b?
N1 N2 N3 N4
Nod
e tim
es in
crea
sing
a
d
a
bb
c
c
d
e e
N5
Partially ordered time structure
Asynchronous Networks – p. 12/60
![Page 34: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/34.jpg)
Examples: computation
Threads need to be synchronized
locks if, for example, other variablesneed to be written.
at each barrier. There may also be
bar
rier
s
Threaded or parallel computation
Locally Synchronous: GALS][Globally Asynchronous +
[Non deterministic process]
[Synchronous]
Single processor computation
Threaded & parallel computation
Asynchronous Networks – p. 13/60
![Page 35: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/35.jpg)
Computation ctd.
Nodes where program is stopped and synchronized
Connection structures – 4 threads
Note: Deadlocks (stop); race conditions (errors)
Asynchronous Networks – p. 14/60
![Page 36: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/36.jpg)
Constrained transport: passing loop
T2 T1
Passing loop (barrier)
Single track line with a passing loop; two trains
Asynchronous Networks – p. 15/60
![Page 37: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/37.jpg)
Constrained transport: passing loop
T1
T1T2
T2
Passing loop (barrier)
Single track line with a passing loop; two trains
Asynchronous Networks – p. 16/60
![Page 38: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/38.jpg)
Constrained transport: passing loop
T1
T1T2
T2
Passing loop (barrier)
Single track line with a passing loop; two trains
Issues:Deadlocks (or livelocks: convergence to blocking
attractor).Logic
Note: Order of entry into passing loop irrelevant.Asynchronous Networks – p. 16/60
![Page 39: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/39.jpg)
Passing loop, variation
T2 T1
T3
S1S2 S3
Passing loop
Single track line with a passing loop and branch; three trains
T1 terminates atS1; T2 atS2; T3 atS3.Unlike in the previous case,order of entry of trainsinto loop is critical– asynchronous logic is nowfragile: an error results in a deadlock or race (ifT2, T3
attempt to enter loop at same time). Simple model –but very widely applicable.
Asynchronous Networks – p. 17/60
![Page 40: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/40.jpg)
Spiking neuron models; switching
C
B
A
0
0
NodesA, B evolve (continuous dynamics). If state ofeitherA orB reaches a threshold, then node fires – aspike or pulse – towards target nodeC (stopped).
Asynchronous Networks – p. 18/60
![Page 41: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/41.jpg)
Spiking neuron models; switching, ctd
C
B
A
0
1
NodeB fires a spike towardsC – receipt registered bychanging input state to1. NodesA andB continue toevolve, NodeC stopped.
Asynchronous Networks – p. 19/60
![Page 42: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/42.jpg)
Spiking neuron models; switching, ctd
C
B
A
1
1
NodeA fires a spike towardsC – Both inputs ofCare now activated.
Asynchronous Networks – p. 20/60
![Page 43: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/43.jpg)
Spiking neuron models; switching, ctd
C
B
A
1
1 ~
Various possibilities: (A) (Shown) With both inputsfilled, C starts and further inputs blocked (inputs setto zero after fixed time, or decay to zero LIF).
Asynchronous Networks – p. 21/60
![Page 44: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/44.jpg)
Spiking neuron models; switching, ctd(B) Order of filling inputs may matter:C only starts ifB firesbeforeA. Similar to passing loop with branchexample or in distributed production systems. Orderthat parts/chemicals/signals are received may becritical for functionality.
Asynchronous Networks – p. 22/60
![Page 45: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/45.jpg)
Spiking neuron models; switching, ctd(B) Order of filling inputs may matter:C only starts ifB firesbeforeA. Similar to passing loop with branchexample or in distributed production systems. Orderthat parts/chemicals/signals are received may becritical for functionality.
In large complex systems, the asynchronous logic(handshaking protocols) involved in running a systemwhere order of inputs matters is likely to make thesystem very fragile and susceptible to deadlocks (egtimetable disruption).
Asynchronous Networks – p. 22/60
![Page 46: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/46.jpg)
Spiking neuron models; switching, ctd(B) Order of filling inputs may matter:C only starts ifB firesbeforeA. Similar to passing loop with branchexample or in distributed production systems. Orderthat parts/chemicals/signals are received may becritical for functionality.
In large complex systems, the asynchronous logic(handshaking protocols) involved in running a systemwhere order of inputs matters is likely to make thesystem very fragile and susceptible to deadlocks (egtimetable disruption).
Adaptation and randomness are likely to play a majorrole in any complex asynchronous network; inparticular, to avoid deadlocks. Note that spikes avoidrace conditions “a.s.”.
Asynchronous Networks – p. 22/60
![Page 47: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/47.jpg)
Asynchronous NetworksA general definition is given in terms of events –which may be deterministic or stochastic – and localtimes (in the non-autonomous case) and continuous ordiscrete dynamics. We give a formal definition in thesimplest case: discrete, deterministic and autonomous.
Asynchronous Networks – p. 23/60
![Page 48: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/48.jpg)
Asynchronous NetworksA general definition is given in terms of events –which may be deterministic or stochastic – and localtimes (in the non-autonomous case) and continuous ordiscrete dynamics. We give a formal definition in thesimplest case: discrete, deterministic and autonomous.
Assume a fixedN set of nodes:N0, · · · , Nn (N0
denotesnull node).
Asynchronous Networks – p. 23/60
![Page 49: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/49.jpg)
Asynchronous NetworksA general definition is given in terms of events –which may be deterministic or stochastic – and localtimes (in the non-autonomous case) and continuous ordiscrete dynamics. We give a formal definition in thesimplest case: discrete, deterministic and autonomous.
Assume a fixedN set of nodes:N0, · · · , Nn (N0
denotesnull node).
Let C denote the set of all directed connectionstructures onN (no self- or multiple connections).Note that∅ ∈ C.
Asynchronous Networks – p. 23/60
![Page 50: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/50.jpg)
Asynchronous NetworksA general definition is given in terms of events –which may be deterministic or stochastic – and localtimes (in the non-autonomous case) and continuous ordiscrete dynamics. We give a formal definition in thesimplest case: discrete, deterministic and autonomous.
Assume a fixedN set of nodes:N0, · · · , Nn (N0
denotesnull node).
Let C denote the set of all directed connectionstructures onN (no self- or multiple connections).Note that∅ ∈ C.
Fix a non-empty subsetA of C. EveryC ∈ A givesN the structure of a directed graph (the connectionmatrix is a01matrix with diagonal elements zero).
Asynchronous Networks – p. 23/60
![Page 51: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/51.jpg)
Asynchronous Networks ctdAssume each nodeNi, i 6= 0, has associated phasespaceMi. SetM =
∏ni=1Mi
Asynchronous Networks – p. 24/60
![Page 52: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/52.jpg)
Asynchronous Networks ctdAssume each nodeNi, i 6= 0, has associated phasespaceMi. SetM =
∏ni=1Mi
Assume that eachC ∈ A determines a smooth(enough) mapfC : M→M satisfying
• For i ∈ {1, · · · , N}, j 6= i, f iC
depends onxj ∈ Nj only if there is an edgeNj→Ni in C.
• If there is an edgeN0→Ni, thenf iC
is constant(stopped node).
Asynchronous Networks – p. 24/60
![Page 53: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/53.jpg)
Asynchronous Networks ctdAssume each nodeNi, i 6= 0, has associated phasespaceMi. SetM =
∏ni=1Mi
Assume that eachC ∈ A determines a smooth(enough) mapfC : M→M satisfying
• For i ∈ {1, · · · , N}, j 6= i, f iC
depends onxj ∈ Nj only if there is an edgeNj→Ni in C.
• If there is an edgeN0→Ni, thenf iC
is constant(stopped node).
Assume given anevent mapE : M→A.
Asynchronous Networks – p. 24/60
![Page 54: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/54.jpg)
Asynchronous Networks ctdAssume each nodeNi, i 6= 0, has associated phasespaceMi. SetM =
∏ni=1Mi
Assume that eachC ∈ A determines a smooth(enough) mapfC : M→M satisfying
• For i ∈ {1, · · · , N}, j 6= i, f iC
depends onxj ∈ Nj only if there is an edgeNj→Ni in C.
• If there is an edgeN0→Ni, thenf iC
is constant(stopped node).
Assume given anevent mapE : M→A.
This data defines the structure of a discreteasynchronous network – synchronous if|A| = 1 .
Asynchronous Networks – p. 24/60
![Page 55: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/55.jpg)
DynamicsGiven data for a discrete asynchronous network asabove, we defineF : M→M by
F (X) = (f 1E(X)(X), · · · , fn
E(X)(X)), X ∈ M.
Provided the event map is not constant (synchronouscase) and we avoid trivial cases (eg the mapsfC areidentical), the operatorF will not be analytic(switching isforcedin asynchronous networks).
In practice, we add conditions to avoid degeneracies.In many situations (eg passing loop), the event mapwill be constant on an open dense set.
An example of astate dependentdynamical system(engineers terminology).
Asynchronous Networks – p. 25/60
![Page 56: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/56.jpg)
Examples
• Random connection structure (RDD network).◮
• Adaptive network and sloppy asynchronouslogic. ◮
• STDP in a spiking neural network.◮
Asynchronous Networks – p. 26/60
![Page 57: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/57.jpg)
Dynamics, Example I1. Random (in time) connection structure.
2. Discrete phase oscillator dynamics.Inhomogeneous ‘Poisson neuron’ firing model:probability of a node firing is state dependent.
p(θ) = 16θ2(1− θ)2, Bell.
p(θ) =
0.05, θ ≤ 0.5− d,
0.05, θ ≥ 0.5 + d,
0.95, θ ∈ (0.5− d, 0.5 + d)
Pulse
1
0Logistic Cubic Bell Pulse
Asynchronous Networks – p. 27/60
![Page 58: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/58.jpg)
MapsSetN = {1, · · ·N}. Fix ωi > 0, i ∈ N and constantsa, b, c ∈ R. For i 6= j ∈ k, θ ∈ T
N , define
Fij(θ) = aSin(θi − θj) + bSin(2(θi − θj + c))
As iterative scheme, take
θn+1i = θni + ωi +
1
k
∑⋆
jFji(θ
n)
where the sum is over allj such that cellj fired andthere is a connectionj→i.
This system can be modelled as a place dependentRDS (the number of symbols grows super-exponentially fast inN : ∼ 2N
2
).Asynchronous Networks – p. 28/60
![Page 59: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/59.jpg)
Visualization of dynamicsUse a system of contractive cocycles forced by(firing) dynamics.
If the system hasN nodes, regard the nodes asvertices of a regular polygon, centered at the origin ofR
2 ≈ C. Denote the coordinates ofCj byZj.
Associated to the nodeCj we define a contractionmappingfj with fixed pointZj by
fj(z) =1
2(z + Zj).
Take the initial pointz0 = 0 ∈ C.
Asynchronous Networks – p. 29/60
![Page 60: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/60.jpg)
MeasurementSuppose constructed the sequencez0, z1, . . . , zn afterm ≥ n time steps. At the(m+ 1)th step of theiteration, suppose that the nodesCj1, . . . , Cjk fire (ifno nodes fire, do nothing, go to the next iteration).Define
zn+1 =1
k
k∑
i=1
fji(zn).
At least numerically,(zn) converges (often slowly) toan attractor with associated invariant measure (andusuallyDN symmetry!). The attractor and measurereflect statistical properties of the node dynamics (egstatistics of synchrony patterns).
Asynchronous Networks – p. 30/60
![Page 61: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/61.jpg)
8-node example: Pulse probability
Visualization of clustering and synchronization: randomconnection structure,ω = 0.0001, a = 0.1301, b = −0.15, c = 0.
Asynchronous Networks – p. 31/60
![Page 62: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/62.jpg)
8-node example: Bell probability
ω = 0.0002, a = 0.16, b = −0.0336, c = 0.Asynchronous Networks – p. 32/60
![Page 63: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/63.jpg)
Example A
ω = 0.0002, a = 0.06, b = −0.0336, c = 0.Asynchronous Networks – p. 33/60
![Page 64: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/64.jpg)
Example B
ω = 0.0002, a = 0.06, b = −0.0336, c = 0.Asynchronous Networks – p. 34/60
![Page 65: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/65.jpg)
Example C
ω = 0.0002, a = 0.06, b = −0.0336, c = 0.Asynchronous Networks – p. 35/60
![Page 66: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/66.jpg)
Invariant subspacesWe recall that the map used for these examples is
θi 7→ ω+θi+1
k
∑⋆
0.06 sin 2π(θj −θi)−0.0336 sin 4π(θj −θi),
where the sum is over thek ‘fired’ cells connected totheθi-cell andω = 0.00002.
In this case the cell states synchronize intoeither twoclusters of4 cells,or one cluster of5 and one of3cells. So eitherθj = θi or θj − θi = α, where
0.06 sin 2πα− 0.0336 sin 4πα = 0.
(Henceα = 12π cos
−1(0.89285) = 0.074.)
Asynchronous Networks – p. 36/60
![Page 67: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/67.jpg)
Intermingled basins of attractionThe invariant4 : 4, 3 : 5 subspaces defined byθj − θi = 0, α can be shown to be normally hyperbolicattracting (neutral stabilities in the subspace, phaseshift directions). Given any initial point, there isalmost sure convergence to one of the252 differentattractors corresponding to4 : 4 or 5 : 3 clustering.
More precisely, letE denote the set of4 : 4, 3 : 5subspaces. Forx0 ∈ T
8, ω(x0) exists a.s. Define
B0 = {x0 ∈ T8 | ω(x0) ⊂ ∪E∈EE},
B1 = {x0 ∈ T8 | ∃!E ∈ E , ω(x0) ⊂ E}.
Asynchronous Networks – p. 37/60
![Page 68: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/68.jpg)
Intermingled basins of attraction ctd.
That is, ifx0 ∈ B1, ω(x0) is always subset ofsameE.Forx0 ∈ B0, we may get differentE each timeiteration is run (case of intermingled basins ofattraction).
µ(B0) = 1, B0 6= T8 and0 < µ(B1) < 1.
Asynchronous Networks – p. 38/60
![Page 69: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/69.jpg)
Intermingled basins of attraction ctd.
That is, ifx0 ∈ B1, ω(x0) is always subset ofsameE.Forx0 ∈ B0, we may get differentE each timeiteration is run (case of intermingled basins ofattraction).
µ(B0) = 1, B0 6= T8 and0 < µ(B1) < 1.
One way of breaking the invariant subspace structureis by using the termsin(4π(θj − θi − c)), c 6= 0, ratherthansin(4π(θj − θi)). Alternatively, we may assumethatω = ωi (say with uniform distribution in[ω − δ, ω + δ], 0 < δ/ω ≪ 1). We show a movie ofthe result (either case).◭ ◮
Asynchronous Networks – p. 38/60
![Page 70: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/70.jpg)
Dynamics, Example 2We want to address the problem of asynchronouslogic in large asynchronous networks. We present anexample of a synchronous adaptive network as anillustration of one way to overcome the problem of thefragility of and complexity of asynchronous logic.Two of the illustrations we present are reallyasynchronous.
1. All-to-all connection structure.
2. Node dynamics given by odd logistic maps.
fλ(x) = λx(1− x2).
3. Adaptive network – spatial verion ofSpike-Timing Dependent Plasticity (STDP).
Asynchronous Networks – p. 39/60
![Page 71: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/71.jpg)
Adaptive network of odd logistic maps
We assumeN nodes whereN ∈ [2, 104] and nodedynamics given odd-logistic maps. We rescale to[0, 1]and take
Fλ(x) =λ
2(1− 18x+ 48x2 − 32x3) +
1
2, λ ∈ [−1, 1]
Denote weight of connection from nodej to nodei bywij and assumewij ∈ [0, 2]. State update rule given by
xn+1i = Fλn(xni ) +
α
NWi(x
n),
whereWi(xn) =
∑
j 6=iwnijx
nj . In our example, we
takeα = 0.45
Asynchronous Networks – p. 40/60
![Page 72: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/72.jpg)
Weight update ruleIf at timen states and weights are given byxni , wn
ij,then
wn+1ij = max{0,min{2, wn
ij +∆(wnij)}},
where∆(wn
ij) = F (wnij, x
ni , x
nj ),
andF (w, x, y) = G(w)H(x, y). For our example, wetake
G(w) = w, (Multiplicative)H(x, y) = 0.2(1− 4.5min{|x− y|, 1− |x− y|}),
(distance onT).
Asynchronous Networks – p. 41/60
![Page 73: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/73.jpg)
Notes on adaptationObserve the adaptation strengthenswij if |xi − xj| issmall. For example ifxni = xnj , then
wn+1ij = min{2, 1.2wn
ij}.
Conversely if|xi − xj| is large (close to0.5), weightsare weakened. For example, if|xi − xj| = 0.5, then
wn+1ij = 0.75wn
ij.
In the next two slides we show dynamics and weightdynamics over about 5600 iterations for a 6000 nodenetwork.
Asynchronous Networks – p. 42/60
![Page 74: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/74.jpg)
Dynamics: 6000 nodes
Asynchronous Networks – p. 43/60
![Page 75: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/75.jpg)
Weight Dynamics: 6000 nodes
Asynchronous Networks – p. 44/60
![Page 76: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/76.jpg)
Dynamics: 6000 nodes
Asynchronous Networks – p. 45/60
![Page 77: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/77.jpg)
Weight Dynamics: 6000 nodes
Asynchronous Networks – p. 46/60
![Page 78: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/78.jpg)
Dynamics: 6000 nodes
Asynchronous Networks – p. 47/60
![Page 79: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/79.jpg)
Dynamics: 6000 nodes
Asynchronous Networks – p. 48/60
![Page 80: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/80.jpg)
Weight Dynamics: 6000 nodes
◭ ◮ Asynchronous Networks – p. 49/60
![Page 81: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/81.jpg)
Dynamics: STDP
STDP is short forSpike-Timing Dependant Plasticity.
STDP is a mechanism for adaptivity in (biological)networks which depends on relative timings. It is anexample of aHebbianlearning rule (unsupervised orcorrelation based learning):
Cells that fire together wire together
Asynchronous Networks – p. 50/60
![Page 82: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/82.jpg)
Dynamics: STDP
STDP is short forSpike-Timing Dependant Plasticity.
STDP is a mechanism for adaptivity in (biological)networks which depends on relative timings. It is anexample of aHebbianlearning rule (unsupervised orcorrelation based learning):
Cells that fire together wire together
The Barn Owl: Gerstner, Kemptner, Van Hemmen &Wagner,Nature1996. Rapid direction finding towithin 1− 2 degrees by encoding signals requiring atime resolution beyond5µs – an order of magnitudefaster than time constants of owl’s neurons.
Asynchronous Networks – p. 50/60
![Page 83: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/83.jpg)
Dynamics: STDP
STDP is short forSpike-Timing Dependant Plasticity.
STDP is a mechanism for adaptivity in (biological)networks which depends on relative timings. It is anexample of aHebbianlearning rule (unsupervised orcorrelation based learning):
Cells that fire together wire together
The Barn Owl: Gerstner, Kemptner, Van Hemmen &Wagner,Nature1996. Rapid direction finding towithin 1− 2 degrees by encoding signals requiring atime resolution beyond5µs – an order of magnitudefaster than time constants of owl’s neurons.
Proposed mechanism: STDP – based on delays &interaural time differences.
Asynchronous Networks – p. 50/60
![Page 84: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/84.jpg)
STDP: Pattern detectionImage from Masquelier, Guyonneau & Thorpe (2008,PLoS One)
Asynchronous Networks – p. 51/60
![Page 85: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/85.jpg)
Some details
wS T
Assume the neuronS emits spike train
S(t) =∑
δ(t− tSi ),
where· · · < tSi < tSi+1 < · · · .
Asynchronous Networks – p. 52/60
![Page 86: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/86.jpg)
STDP ctdSimilarly assume the neuronT has spike train
T (t) =∑
δ(t− tTi ),
where· · · < tSi < tSi+1 < · · · .
Assume the connection is excitatory (w > 0). Thebasic idea is that ifT fires just afterS, we regard thefiring as having been ‘caused’ byS and increase thecoupling strengthw; if T fires just beforeS, there isno causality and we weaken the coupling strengthw.
More formally, we use a functionW (s) that defines alearning window.
Asynchronous Networks – p. 53/60
![Page 87: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/87.jpg)
STDP ctd
s
W(s)
learning window
(0,0)
Note: usually assume∫
W < 0. If tSi − tTj is in thelearning window, then we changew by
∆(w) = ηH(w)W (tSi − tTj ).Asynchronous Networks – p. 54/60
![Page 88: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/88.jpg)
STDP ctdHereη > 0 (typically η ≪ 1) andH(w) = wµ. IftSi < tTj (causality), then∆(w) > 0.
Assume (simpler) additive case:H(w) = 1. Over alearning session of timeTℓ, we take
∆(w)(t) = η∑
tSi ,tTj ∈I
W (tSi − tTj ),
whereI = [t− Tℓ, t].
One approach to developing a mean field model ofSTDP, due to Burkitt, Gilson, Hemmen et al., is toassume that firings follow an inhomogeneous Poissonstatistic (‘Poisson neurons’):
Asynchronous Networks – p. 55/60
![Page 89: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/89.jpg)
STDP: Mean Field ModelProbability of 1 firing in[t, t+∆t] = λi(t)∆t,Probability of≥ 2 firings in [t, t+∆t] = o(∆t).Firings in disjoint intervals independent.
Under appropriate assumptions on the time scales (egslow learning compared with firing rates and changesin λi(t) small in learning session: adiabatichypothesis) Burkitt et al develop a mean field modelof STDP for quite general recurrent networks ofspiking neurons subject to inputs from Poissonneurons (the latter with fixed Poisson rates). Theirmodel can and does incorporate delays and yields anODE model for evolution of weights.
Asynchronous Networks – p. 56/60
![Page 90: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/90.jpg)
Adaptation & Dynamics DetectionWe consider dynamics detection using STDP.
Q
P
T
Source networks Single node target network
Two source networks connected all-to-1 to a targetnetwork consisting of a single node.
Asynchronous Networks – p. 57/60
![Page 91: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/91.jpg)
DynamicsBoth networksP, Q consist of coupled phaseoscillators — in regimes where the oscillators willeventually frequency synchronize or do somethingelse “interesting”...
Each time a node state passes through1, the oscillatorfires a spike.
All oscillators are connected to target node – eachconnection has weightw ∈ [0, 1].
Sum inputs intoT. If sum exceeds a threshold,T
fires and its state is reset to0. (Various protocolsallowed: SRM_0, SRM & gated.)
Asynchronous Networks – p. 58/60
![Page 92: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/92.jpg)
Adaptation: STDPWe adapt weights according to STDP.
If T fires (shortly) after a nodeN ∈ P ∪Q fires, weregard the firing ofN as having caused the firing ofTand strengthen the weight of the connection betweenN andT. Conversely ifT fires (shortly) before anodeN ∈ P ∪Q fires, we regard the events asuncorrelated and weaken the weight of the connectionbetweenN andT.
This form of adaptation is called Spike-TimingDependent Plasticity in computational neuroscience.
We illustrate with some numerical examples.◭ ◮
Asynchronous Networks – p. 59/60
![Page 93: Dynamics on Asynchronous Networks](https://reader030.fdocuments.us/reader030/viewer/2022012722/61b4557f0a207f16081bb17a/html5/thumbnails/93.jpg)
Mathematical challenges1. How does asynchronicity impact dynamics?
2. How do we analyze without the assumption that equations
are analytic?
3. Bifurcation theory in adaptive spiking networks (STDP) –
many interesting questions and phenomena already.
4. Understanding how & why asynchronous networks can
work correctly (most of the time) notwithstanding the
fragility and complexity of asynchronous logic. On the
neuro-computation side, the basic mechanisms may not be
so hard to understand – evolution can lend a helping hand.
With stochastic asynchronous networks, analysis may be
much easier than in the deterministic case! Applications to
‘Qualitative Computing’.Asynchronous Networks – p. 60/60