Modeling Neural Networks
description
Transcript of Modeling Neural Networks
Modeling Neural Networks
Christopher Krycho
Advisor: Dr. Eric Abraham
May 14, 2009
Computational Neuroscience
• Young field (decades old)• Modeling the brain with
physical principles– Oscillatory behaviors– Large-scale networks
• Technique:– Create a model of neurons– Model neuron interactions
(varying degrees complexity)– Compare results to
EEG/MRI/etc.An EEG recording of
various locations in a brain
Project Goals• Build a computer program that builds and
analyzes small-world networks• Apply that model to neural network models
created by the Zochowski group at Michigan– Reproduce neuron and network models– Run a simulation with that model and compare
results
• Test different models and compare results– Doing new science by examining variations on
their model
Outline• What are neurons and neural networks?• Small world networks and their relevance• Modeling a set of neurons in the brain with
small world networks• Graph, analyze total synaptic current in
system over time.
Neurons• Primary components of nervous system• Transmission and reception of electrical signals
– “synaptic” current: current along synapses - connections between neurons
Representation of a neuron
Source: Wikipedia
The human nervous system
Source: http://sciencecity.oupchina.com.hk/biology/
The Nervous System• Composed of neurons
throughout– Brain– Nerves in fingers– Spinal cord
• 1011 neurons in the brain alone– Each with 105+ connections to
other neurons (and itself)
Small World Networks (SWNs)• Small average “path length”
– Number of steps from random node to other random node
• Large number of nodes– # of nodes in network
• Low connectivity– # of connections per node
• High clusteredness– Nodes with mutual
connections also connectto each other
A network with small world characteristics
Source: Exploring Complex Networks, Steven H. Strogatz, Nature 410 268-276 (8 March 2001)
Network vs. SWN• “Small world” different from random networks• Despite…
– low relative connectivity– Connections primarily short-range– Connections very clustered
• … short average path length between any random points in network
• SWN architecture observed in brain– On a small scale in local neural networks– Possibly on a larger scale among neural networks
Making SWNs• Initialize an array representing neurons’
connections to each other– Connections initially symmetric within networks– Two or more networks connected internally, but
not to each other– Note: actual networks also connected to next-
nearest neighbors
Two networks of 9 nodes each:
A
B
From network to SWN, pt. 1• Randomly rewire the internal connections of
each network– Use a random number generator– Connections can be symmetric or directional
• Here, symmetric• In model, directional
The networks after internal rewiring:
AB
From network to SWN, pt. 2• Randomly add connectivity between networks
– Choose some proportion of the neurons in any given network to receive connections
– Choose the number in any given network sending connections
– Connect - showing directional connections here
The networks after adding connectivity:
Key:
From A to B
From B to A
A
B
Representing Connections• Matrices filled with 0s and 1s
– Rows send, columns receive
1 1
1
1 2
2 2 2
A small network rewired
• From left to right:– The initial setup of the network - symmetric– The networks, with random rewiring of 0.3 of connections– 25% of neurons on each network receiving input from 4
different neurons on the other network
The networks before any rewiring:
Representative Connection Graphs
• 15x15 grids = 225 neurons/network
• 2 networks• 450x450 matrix:
– Rows send
– Columns receive
• Connections to nearest and next-nearest neighbors
Rep. Conn. Graphs cont’d.
The networks after internal rewiring:
Rep. Conn. Graphs cont’d.
The networks after adding internetwork
connections:
A Neuron Model• Equation representing the state of each neuron at a
given time– Numerical value representing charge (V) on the neuron
and thus closeness to “spiking” (sending a signal)
• Equation involves 4 parameters:
i - Leakage current on ith neuron; constant over time but differs for each neuron
Ji,j(t) - Incoming current to the ith neuron from the jth neurons (connected, in the same network)
Ji,k(t) - Incoming current to the ith neuron from the kth neurons (connected, in another network)
(t) - White noise
€
dVidt
= −α iV t( ) + A Ji, j t( )j
∑ + B Ji,k t( )k
∑ +ξ t( )
Generating Current cont’d.• Equation (identical form for intra/internetwork):
• Ji,j(t): current to neuron i from neuron j at time t
– Ji,k(t) has the same form, but different lag
s and f ensurecorrect shapeof pulse
€
Ji, j (t) = exp −t
τ s
⎛
⎝ ⎜
⎞
⎠ ⎟− exp −
t
τ f
⎛
⎝ ⎜ ⎜
⎞
⎠ ⎟ ⎟
Current versus time; threshold reached at t=0
Solving the Equation• Integrate over 10 sec with Euler numerical
method– Maximum 10-3 sec step size for good resolution
(some runs as low as 10-5 sec)
• Large computation:– 450 neurons – 5 calculations/neuron/iteration– ~ 225 billion calculations at highest resolution
• ~ 45-50 minutes of real time for high resolution– Coded in Fortran– Running on 2007 MacBook Pro
Network Progress
Network Finished
Simulation Progress• Replicating Zochowski model: in progress
– Reproducing underlying phenomena– Missing the tell-tale sign of perfectly reproducing their model:
“bursting”
The Zochowski group’s model graph of network
behavior:
Early Simulation Run
Time (s)
To
tal s
yna
ptic
cu
rren
t (ar
b. U
nits
)
Network 2
Network 1
Present Simulation Run
The Zochowski group’s model showing bursting:
Our model showing basic behaviors but no bursting
Result Details
The Zochowski model individual neurons
Our model individual neurons
The Future (of the Project)• Finish replicating Zochowski group’s results• Open doors for the future• Possibility of expanding the number and/or
scale of the networks• My design includes
– Ability to implement and then test learning mechanisms
– Ability to increase complexity of neuron model– Ability to increase complexity of network model
Acknowledgments• The Zochowski group at The University of
Michigan, particularly Jane Wang and Sarah Feldt, for their work on which this project is based, and for answering many questions along the way
• Dr. Keiran Mullen for teaching me enough Fortran to start the project
• Dr. Eric Abraham, my advisor