Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC...
Transcript of Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC...
![Page 1: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/1.jpg)
Computational Neuroscience ESPRC Workshop--Warwick
Information-GeometricStudies on Neuronal SpikeTrains
Shun-ichi AmariShun-ichi AmariRIKEN Brain Science Institute
Mathematical Neuroscience Unit
![Page 2: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/2.jpg)
Neural Firing
1x 2x3
xn
x
higher-order correlations
orthogonal decomposition
1 2( ) ( , ,..., ) : joint probabilityn
p p x x x=x
[ ]i ir E x=
[ , ]ij i jv Cov x x=
----firing rate
----covariance: correlation
1,0i
x =
1 2{ ( , ,..., )}n
S p x x x=
![Page 3: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/3.jpg)
( ) 0,1i
x t =
time tneuron i
1 T
1 ( )11x
1( )x T
( )21x ( )2
x T
n ( )1n
x ( )nx T
Multiple spike sequence:
{ ( ), 1,..., ; 1,..., }i
x t i n t T= =
![Page 4: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/4.jpg)
( ){ } ( )( )
2
2
1; , ; , exp
22
xS p x p x
μμ μ= =
Information GeometryInformation Geometry ? ?
( ){ }p x
μ
( ){ };S p x= è
Riemannian metric
Dual affine connections
( , )μ=è
![Page 5: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/5.jpg)
Information GeometryInformation Geometry
Systems Theory Information Theory
Statistics Neural Networks
Combinatorics PhysicsInformation Sciences
Riemannian Manifold
Dual Affine Connections
Manifold of Probability Distributions
Math. AI
![Page 6: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/6.jpg)
Manifold of Probability DistributionsManifold of Probability Distributions
( )1 2 3 1 2 3
1, 2,3 { ( )}
, , 1
x p x
p p p p p p
=
= + + =
3p
2p
1p
p
( ){ };M p x=
![Page 7: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/7.jpg)
Manifold of Probability DistributionsManifold of Probability Distributions
( ){ },
1, 2,3
P x
x
=
=
M p
3
21
( ) ( ) ( )3
3 1 2
1
, 1i i
i
P x p x p p p=
= =p
3
21
M
( )1 2 3
1 2 3
, ,
1
p p p
p p p
=
+ + =
p
2 2 2
1 2 31
i ip=
+ + =
( )
11
3 1 2
22
3
log
,
log
p
p
p
p
=
=
=
![Page 8: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/8.jpg)
InvarianceInvariance ( ){ },S p x=
1. Invariant under reparameterization1. Invariant under reparameterization
( )( , ) ,p x p x=
2. 2. Invariant under different representationInvariant under different representation
( ) ( ), { , } { ( , )}y y x p y p x= = ( ) ( )2
1 2
2
1 2
, ,
| ( , ) ( , ) |
p x p x dx
p y p y dy
2 2
i iD =
![Page 9: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/9.jpg)
Two StructuresTwo Structures
Riemannian metricRiemannian metric
affine connection --- geodesicaffine connection --- geodesic
( )2,ij i jds g d d d d= =< >
Fisher informationFisher information
log logij
i j
g E p p=
![Page 10: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/10.jpg)
Affine Connection
covariant derivative
geodesic X=X X=X(t)
( )
c
i j
ij
X Y
s g d d
=
=
minimal distance
& &
straight line
![Page 11: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/11.jpg)
Affine Connectioncovariant derivative; parallel transport
,
geodesic =x x=x(t)
:
, ,
( )
X c
i j
i
i j
ij
j
Y X Y
x
s g d
X Y X
d
Y g X Y< >=< >
=
=
minimal distance straight line
& &
![Page 12: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/12.jpg)
Duality: two affine connectionsDuality: two affine connections
, , ,i j
ijX Y X Y X Y g X Y= < >=
Riemannian geometry:Riemannian geometry: =
X
Y
X
Y
*
{ , , , *}S g
![Page 13: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/13.jpg)
Dual Affine ConnectionsDual Affine Connections
e-geodesice-geodesic
m-geodesicm-geodesic
( ) ( ) ( ) ( ) ( )log , log 1 lgr x t t p x t q x c t= + +
( ) ( ) ( ) ( ), 1r x t tp x t q x= +
( ),
( )q x
( )p x
*( , )
( ) 0
* ( ) 0
x
x
x t
x t
=
=
&
&
&
&
![Page 14: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/14.jpg)
Information GeometryInformation Geometry
-- Dually Flat Manifold-- Dually Flat Manifold
Convex Analysis
Legendre transformation
Divergence
Pythagorean theorem
I-projection
![Page 15: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/15.jpg)
Dually Flat ManifoldDually Flat Manifold
1. Potential Functions1. Potential Functions
---convex (Bregman, Legendre transformation)
2. Divergence2. Divergence [ ]:D p q
3. Pythagoras Theorem3. Pythagoras Theorem
[ ] [ ] [ ]: : :D p q D q r D p r+ =
4.4. Projection TheoremProjection Theorem
5.Dual foliation5.Dual foliation
p
r
q
![Page 16: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/16.jpg)
Pythagoras Theorem
p
q
r
D[p:r] = D[p:q]+D[q:r]
p,q: same marginals
r,q: same correlations
1 2, independent
correlations
( )[ : ] ( ) log
( )x
p xD p r p x
q x=
estimation correlationtesting
invariant under firing rates
![Page 17: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/17.jpg)
Projection Theorem
[ ]min :Q M
D P Q
Q = m-geodesic
projection of P to M
[ ]min :Q M
D Q P
Q = e-geodesic projection of P to M
![Page 18: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/18.jpg)
![Page 19: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/19.jpg)
![Page 20: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/20.jpg)
( ) 0,1i
x t =
time tneuron i
1 T
1 ( )11x
1( )x T
( )21x ( )2
x T
n ( )1n
x ( )nx T
Multiple spike sequence:
{ ( ), 1,..., ; 1,..., }i
x t i n t T= =
![Page 21: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/21.jpg)
spatial correlationsspatial correlations
( )1, ,
nx x= Lx
( ) ( )1 1exp
i i ij i j n n
i j
p x x x x x<
= + + +L
LLx
[ ] { }
{ }Prob 1
Prob 1
i i i
ij i j i j
r E x x
r E x x x x
= = =
= = = =
( )
( )1
1
, , ,
, , ,
i ij n
i ij nr r r
=
=
L
L
L
Lr
L
orthogonal structure
( ){ }S p= x : coordinates
![Page 22: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/22.jpg)
Spatio-temporal correlationsSpatio-temporal correlations
correlated Poisson
( )p x : temporally independent
correlated renewal process
( )tp x : firing rate ( )ir t modified
spatial correlations fixed
![Page 23: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/23.jpg)
firing rates:
correlation—covariance?
1x
2x
3x
00 01 10 11{ , , , }p p p p
1 2 12, ;r r r
Two neurons:
![Page 24: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/24.jpg)
Correlations of Neural Firing
( ){ }{ }
1 2
00 10 01 11
1 1 10 11
2 1 01 11
,
, , ,
p x x
p p p p
r p p p
r p p p
= = +
= = +
11 00
10 01
logp p
p p=
1x 2
x
2
1
1 2{( , ), }r r
orthogonal coordinates
firing rates
correlations
![Page 25: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/25.jpg)
1 2{ ( , )}S p x x=
1 2, 0,1x x =
1 2{ ( ) ( )}M q x q x=
Independent Distributions
![Page 26: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/26.jpg)
Orthogonal Coordinates:
firing rates
r
correlation fixedscale of correlation
r’
![Page 27: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/27.jpg)
two neuron casetwo neuron case
( )
( )( )
( )
( ) ( ) ( )( )
1 2 12 1 2 12
12 12 1 200 11
12
01 10 1 12 2 12
12 1 2
12 1 2
, , ; , ,
1log log
, ,
, ,
r r r
r r r rp p
p p r r r r
r f r r
r t f r t r t
+= =
=
=
![Page 28: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/28.jpg)
Decomposition of KL-divergence
D[p:r] = D[p:q]+D[q:r]
p,q: same marginals
r,q: same correlations
1 2,
p
q
rindependent
correlations
( )[ : ] ( ) log
( )x
p xD p r p x
q x=
![Page 29: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/29.jpg)
pairwise correlations
ij ij i jc r r r=
independent distributions
,,ij i j ijk i j kr r r r r r r= = L
How to generate correlated spikes?
(Niebur, Neural Computation [2007])
higher-order correlations
covariance: not orthogonal
![Page 30: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/30.jpg)
Orthogonalhigher-order correlations
( )
( )1
1
;
;
,
,
,
,
i i n
i n
j
i jrr r
=
=
L
L
L
Lr
![Page 31: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/31.jpg)
tractable models ofdistributions
Full model: parameters
Homogeneous model: n parameters
Boltzmann machine: only pairwise
correlations
Mixture model
2 1n
{ ( )}M p= x
![Page 32: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/32.jpg)
Models of Correlated Spike Distributions (1)
Reference 0110001011 ( )y t
( )1x t%
( )2x t%
( )3x t%
1001011001
0101101101
![Page 33: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/33.jpg)
Models of Correlated Spike Distributions (2)
( ) ( ) ( )i ix t x t y t= % o
additive interaction
eliminating interaction
Niebur replacement
( )y t
( )1
2 ( )
x t
x t
%
%
0110001011
1001001101
1100010011
![Page 34: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/34.jpg)
Mixture Model
( ):y t 0110001011 mixing sequence
( ) ( )1; , when 1
i ip p x u y= =x u
( ) ( )2; , when 0
i ip p x v y= =x u
( , ) : 1 with probability p x u x u=
( ) ( ) ( | )p p y p y=x x
![Page 35: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/35.jpg)
Mixture modelMixture model
( ) ( )
( ) ( ) ( )
; ,
; , , ; ;
i ip p x u
p m mp mp
=
= +
x u
x u v x u x v
( )
( )1
1
, , 1
, ,
n
n
u u m m
v v
= =
=
L
L
u
v
( ){ } ( ){ }; , , ,n n
M p m S p= =x u v x
: independent distribution
![Page 36: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/36.jpg)
firing ratesfiring rates
ij i j i jr mu u mv v= +
ijk i j k i j kr mu u u mv v v= +
i i ir mu mv= +
L
![Page 37: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/37.jpg)
Conditional distributionsand mixture
1,2,3,...
( | ) : Hebb assembly
( ) ( ) ( | )
y
p y y
p p y p y
=
=
x
x x
![Page 38: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/38.jpg)
State transition of Hebb assembliesmixture and covariances
( ) ( ) ( ) ( ), 1u v
p t t p tp= +x x x
( ) ( ) ( ) ( ) ( )( )( )1 1ij ij ij i i j j
c t t c u tc v t t u v u v= + +
Extra increase (decrease) of covariances
within Hebb assemblis---- increase between Hebb assemblies ---decrease
![Page 39: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/39.jpg)
Temporal correlations {x(t)}, t =1, 2, …T
Poisson sequence:
Markov chain
Renewal process
Prob{ (1),..., ( )}x x T
![Page 40: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/40.jpg)
Orthogonal parameter ofcorrelation ---- Markov case
1
1
1 1
( | )
Pr{ | }
Pr{ } Pr( ) ( | )
t t
ij t t
t t t
p x x
a x i x j
x x p x x
+
+
+
= = =
=
1
2
11 1
11 00
10 01
Pr{ 1}
log
tr x
c r r
a a
a a
= =
=
=
![Page 41: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/41.jpg)
Spatio-temporal correlation
correlated Poisson
1({ }) ( )
T
t tt
p p=
=x x
![Page 42: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/42.jpg)
Correlated renewal process
,
Pr( ) ( ') : ' last spike time
Pr( ) ( ') ( | )
t
i t i
x k t t t
x k t t p x
=
= x
![Page 43: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/43.jpg)
Neurons
1x
nx
( )1i i
x u=
Gaussian [ ]i i j
u E u u= =
2x
Population and Synfire
![Page 44: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/44.jpg)
Population and Synfire
hswu jiji =
[ ]ii
ux 1=
(1 )i i
u h= +
( ), 0, 1i
N
s
1x
nx
2
[ ]
[ ] 1
i j
i
E u u
E u
=
=
![Page 45: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/45.jpg)
{ } timesame at the fire neurons Prob ipi=
Pr{ neurons fire}r
ir P nr
n= =
( ) ( )( , )
nH r nzq r e e d=
( ) ( ) ( )FrFrn
z = 1 log 1 log 2
2
( ) dt 2
1 2
0
2t
ha
ehaFF ==
![Page 46: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/46.jpg)
1 22 1( , ) exp[ { ( ) } ]
2(1 ) 2 1q r c F h=
1 2
1
...
( , ) exp{ ...}
(1/ )k
i i ij i j ijk i j k
k
i i i
p x x x x x x
O n
= + + +
=
x
![Page 47: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/47.jpg)
Synfiring
( )
1( ) ( ,..., )
1
n
i
p p x x
r x q rn
=
=
x
( )q r
r
![Page 48: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/48.jpg)
Bifurcation
r
rP
ix : independent---single delta peak
pairwise correlated
higher-order correlation
![Page 49: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/49.jpg)
Semiparametric StatisticsSemiparametric Statistics
( ){ }( ) ( )
, ,
,
M p x r
p x r x
=
=
y x=
'
i i i
i i i
y
x
= +
= +
mle, least squre, total least square
( ) ( ) ( ), ; , ; ,p x y p x y r d=
x
y
![Page 50: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/50.jpg)
Linear Regression: Semiparametrics
( )
( )
( )
1 1
2 2
,
,
,n n
x y
x y
x y
M
'
i i i
i i i
x
y
= +
= +
( )' 2, 0,
i iN
y x=
![Page 51: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/51.jpg)
Statistical Model
( ) ( ) ( )
( )
( ) ( ) ( )
2 2
1
1 1, , exp
2 2
, , : , , ,
, , ,
i i i n
p x y c x y
p x y
p x y p x y Z d
=
=
L
semiparametric
![Page 52: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/52.jpg)
Least squares?
( ) ( )
( )( )
2
2ˆmin :
1,
mle, TLS
0
Neyman-Scott
i i
i i
i
ii
i i
i i i i
x yL y x
x
yy
n x x
y x y x
= =
+ =
![Page 53: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/53.jpg)
( )
( ) ( )
( )
( )
'
1 2
,
,
, , , ,
, , 0
, 0
ˆ, 0
Z
Z
i
x x p x Z
y x E y x
E y x
y x
=
=
Semiparametric statistical model
Estimating function
Estimating equation
L
![Page 54: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/54.jpg)
( )
( ) ( )
( ){ }( )
( )( )
, ; , log
, ; ,
, ;
, ,I
u x y Z p
sE s
v x y Z E f s
k s x y
u x y u E u s
k x y y x
=
=
=
=
=
= +
L
![Page 55: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/55.jpg)
Fibre bundreFibre bundre
function spacefunction space
r
![Page 56: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/56.jpg)
( ) ( )( ) ( )( )
2
2
,
, ,I
z N
u x Z x y c y x
c
μ
μ
= + +
=
( ) ( )( )
( )
2
222 2
, ;
1
1
i
i
f x c x y c y x
c
xn
xn
μ
μ
μ
= + +
=
=
=
![Page 57: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/57.jpg)
Poisson process
Poisson Process: Instantaneous firing rate is constant over
time. dtFor every small time window dt,
generate a spike with probability dt.
T
Cortical Neuron Poisson Process
Poisson processPoisson process
cannot explain inter-cannot explain inter-
spike intervalspike interval
distributions.distributions.
TeTp =)(
(Softky & Koch, 1993)
![Page 58: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/58.jpg)
Gamma distribution
Gamma Distribution: Every -th spike of the Poisson process is left.
: Firing rate
: Irregularity.T
)(
)(),q(T;
T1= e Two parameters
=1 (Poisson)
=3
T
![Page 59: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/59.jpg)
Gamma distributionGamma distribution
( )( )( )
{ }1exp
rf T T r T=
1=
Integrate-and fire
Markov model
: Poisson
: regular
![Page 60: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/60.jpg)
Irregularity is unique to
individual neurons.
t
t
t
Regular large
Irregular small
Irregularity varies among
neurons.)
We assume that We assume that is independent of is independent oftime.time.
![Page 61: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/61.jpg)
Information geometry of
estimating function•Estimating function y(T, ):
(See poster for details)
)(
),(log),(
),(log),(
îk
kT;êpkT;êv
dê
kT;êpdkT;êu
How to obtain an estimatingfunction y:
•Maximum likelihood Method:
vvv
vuuy
><
><=
Scorefunctions
E[y(T, )]=0
0);(1
==
N
l
lTy0);()()(log
1
1 ===
N
l
lN TuTpTpd
dL
![Page 62: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/62.jpg)
temporal correlationstemporal correlations
( ) ( ) ( )1 2x x x NL
independent with firing probability ( )r t
spike counts: Poisson
ISI: exponential
renewal:
( ) ( )ir t k t t= : last
spikeISI distribution ( )f t
( ) ( ) ( )0
exp
T
f T ck T k t dt=
![Page 63: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/63.jpg)
Estimation of
by estimating functionsNo estimating function exists if the neighboring firing rates aredifferent.
m( ) consecutive observations must have the same firing rate.
Estimating function:
(E[y]=0)
Example:Example:m=2m=2
( ))(2)2(2log
2
21
21 êöêöTT
TTy +
+=
l-thset:
( )0)(2)2(2log
1
12
21
21 =++
==
êöêöTT
TT
Ny
,m
lll
ll
=0
2121 )(),;(),;(),;,( dkTqTqkTTp
)( 21
ll,TT
Model:
![Page 64: Information-Geometric Studies on Neuronal Spike Trains · Computational Neuroscience ESPRC Workshop--Warwick Information-Geometric Studies on Neuronal Spike Trains Shun-ichi Amari](https://reader034.fdocuments.us/reader034/viewer/2022051809/60105e4ebed2f3676b78ad32/html5/thumbnails/64.jpg)
Case of m=1
(spontaneous discharge)
Firing rate continuously
changes and is driven by
Gaussian noise.: Correlation
( )0)(2)2(2log
1
12
21
21 =++
==
êöêöTT
TT
Ny
,m
lll
ll
can be approximated if can be approximated ifthe firing rate changesthe firing rate changesslowly.slowly.Estimating function (m=2):
slow