Post on 07-Feb-2017
PowerPoint Presentation
A cortical neural network model of visual motion perception for reactive navigation
Cognitive Anteater Robotics Lab (CARL)University of California, IrvineMichael Beyeler
1
December 1, 20152Projects Overview
Understand the neural correlates of visually guided navigationCapture important network dynamicsLink network dynamics to behavior
Build more powerful artificial vision systems and cognitive agentsBehavior and intelligence of organisms with nervous systems surpass any state of the art, man-made systemIncorporate on-line learningIncorporate predictive capabilitiesDecember 1, 20153Long-Term Goal and MotivationPerceptual variablesPhysical variablesSteering system
Reverse engineering the brainUse brain as a working modelUnderstanding through modelingCurrently: Spiking neural networks on GPUsLow-cost yet high-performanceEnables real-time processing of large-scale neural network modelsNear-future: Dedicated neuromorphic hardwareDecember 1, 20154Approach
Spiking Neural Network (SNN) models describe key aspects of neural function and network dynamics
SNN models are composed of:spiking point-neurons for computationIzhikevich spiking neurons: 20 different neuron typesdynamic synapses for learning and memory storageSynaptic receptors for AMPA, NMDA, and GABAvariable-delay axons for communicationneuromodulatory systems to control action selection and learning
December 1, 20155Spiking Neural Networks
(Izhikevich, 2003, 2004)
December 1, 20156Constructing Functional SNN ModelsCARLsim 3
neural circuits:createtune/optimizeexplore
TheoreticalmodelExperimentaldataFunctionalapplication
TheoreticalneuroscienceReal-timeapplicationsNeuromorphicengineering(Nageswaran et al., 2009; Richert et al., 2011, Beyeler et al, 2015)
stop to think about what people are looking for in a simulator: needs to fit your purposes. neuron model, connectivity, plasticity, visualization, tuning.6
GPU-accelerated spiking neural network simulator
User-friendly, well documented.Runs on Linux, Mac OS X, Windows systems with CUDA SDK.Scalable, extendable.PyNN-like interface.Highly optimized for NVIDIA GPUs.Capable of simulating biological detailed neural models.
Freely available at:http://www.socsci.uci.edu/~jkrichma/CARLsim/
December 1, 20157CARLsim in a Nutshell
7
Provides a PyNN-like user interfaceLets you configure networks, apply input stimuli, and monitor network activityDecember 1, 20158CARLsim API
8
Easily create complex network topographies:December 1, 20159Building Biologically Detailed SNNs
Short-term and long-term plasticity:
Versatile toolbox for the visualization and analysis of neuronal, synaptic, and network information.Generates raster plots, histograms, heat maps, flow fields.Plot, record, load, save.December 1, 201510MATLAB Offline Analysis Toolbox
December 1, 201511capable of quickly and efficiently tuning large-scale SNNsarbitrary fitness functionmulti-objective optimization
utilizes Evolutionary Computations in Java (ECJ)fully integrated with CARLsim
transparent use of a GPUup to 60x speedups compared to single-threaded CPU simulationAutomated Parameter Tuning Interface
Benchmark: 80-20 network with E-STDP (Vogels & Abbott, 2005)CPU: Intel Core i7 CPU 920 @ 2.67 GHzGPU: NVIDIA GTX 780 (3 GB of GDDR5, 2304 CUDA cores)
December 1, 201512Performance Benchmarks
Often informed by visual motion cues
Typically requires quick and precise timing
Involves large-scale brain processingReactive Navigation Through a Cluttered Scene
December 1, 201514Primary visual cortex (V1)tuned to simple attributes of shape, motion, color, texture, depth
Middle temporal (MT) areatuned to coherent local motion (retinal flow)
Posterior parietal cortex (PPC)Polysensory areas (VIP, MST, 7a, etc.)Tuned to global, complex motionSelf-motion and object motionSpatial reference framesPath integration (?)Visual Motion Pathway
(Britten, 2008)
December 1, 201515Problem: Local-velocity sample is different from object velocityGoal: Disambiguate local-velocity samples and integrate them into an accurate estimate of the global (object) velocity
Intersection-of-constraints (IOC):Each local velocity sample constrains the global object velocityFind object velocity by integrating local samplesThere is evidence that MT firing rates represent the velocity of moving objects using IOC
Aperture Problem(Bradley & Goyal, 2008)
Motion is an orientation in space-timeV1 simple cells: space-time oriented receptive fields
Spatiotemporal energy model:Linear filtering / motion energy / opponent energyAdelson & Bergen (1985), Simoncelli & Heeger (1998)December 1, 201516Primary Visual Cortex (V1)
(Simoncelli & Heeger, 1998)(Bradley & Goyal, 2008)
December 1, 201517Middle Temporal Area (MT)ExcitationFeedforward and localSpecific InhibitionCross-direction inhibitionUnspecific InhibitionDivisive normalization
MT CDSMT PDS(Beyeler et al., 2015)Based on Rust et al. (2006)
Direction Tuning
Component-direction-selectivePattern-direction-selective
Direction Tuning (2)
V1MT CDSMT PDS(Beyeler et al., 2014)(CDS: component-direction-selective, PDS: pattern-direction-selective)December 1, 201519
Three speed tuning classes observed in MT (Rodman & Albright, 1987)band-pass or speed-tunedlow-passhigh-pass
Stimulus: single bar drifting over entire visual field to the right (preferred) or left (anti-preferred) at different speeds12/1/2015
20Speed Tuning
12/1/2015
21Computational Performance
CPU: single-core Intel Xeon X5675 @ 3.07 GHz (24 GB of RAM)GPU: single NVIDIA Tesla M2090 512 cores @ 1.3 GHz (6 GB of G-DDR5)(Beyeler et al., 2014)
December 1, 201522Android Based Robotics (ABR)
Introducing Le Carl: The French Robot
December 1, 201523Visually Guided Navigation
Behavioral paradigm:Analogous to Fajen & Warren (2003)Navigate towards a distant, visually salient targetAvoid obstacles along the wayOnly sensory input: VisionVisual input:320x240px snapshots~ 20 frames / secondPosition tracking:overhead cameraperspective transform (OpenCV)
December 1, 201524Setup and Network Architecture(Beyeler et al., 2015)40,000 neurons1.7 million synapses
December 1, 201525Inspired by the Balance Strategy:Simple control law for obstacle avoidanceUsed by honeybees for visual control of flight (Srinivasan & Zhang, 1997)Humans might make use of similar strategy (Kountouriotis et al., 2013)
Important difference:Obstacles are located via motion discontinuitiesPosterior Parietal Cortex (PPC)
December 1, 201526ABR Cockpit View
December 1, 201527Simulated Neuronal Responses(Beyeler et al., 2015)
December 1, 201528Behavioral Results
Comparison to Psychophysical Data
December 1, 201529
CARLsim highlights:Runs on Linux, Mac OS X, Windows.Highly optimized for NVIDIA GPUs: CUDA 5.0, device capability 2.0User-friendly, scalable, extendableDownload: www.socsci.uci.edu/~jkrichma/CARLsim
Presented a large-scale spiking neural network thatis biologically inspiredsolves the aperture problem via cortical mechanismsis integrated with a real-time, real-world robotics platformAndroid Based Robotics combined with CARLsim is the first step toward a complete robot navigation system.December 1, 201530Conclusions
December 1, 201531Other Projects
(Beyeler et al., 2013)
Add MNIST, add backup slides31
December 1, 201532Team CARL (2015)Front row Emily Rounds, Steve Doubleday, Jeff Krichmar, Ting-Shuo Chou, Nikil DuttBack row Alexis Craig, Alex Wang, Michael Beyeler, Feng Rong, Timo Oess, Saideep GuptaSupported by the National Science Foundation and Qualcomm Technologies Incorporated.
Update picture32
December 1, 201533
12/1/2015
34Izhikevich Spiking Neurons
(Izhikevich, 2003)
12/1/2015
35Spiking Neural Network Models
(Izhikevich, 2004)
December 1, 201536Motion Energy Model
(Adelson & Bergen, 1985;Simoncelli & Heeger, 1998)
December 1, 201537Solving the Aperture Problem
(Bradley & Goyal, 2008)
Vector average (VA)Advantage: simpleVA is a poor predictor of object direction and speedVA speed almost always less than object speedVA tends to depend on object shapeAssumes symmetrical distribution of local-velocity samplesDecember 1, 201538Solving the Aperture Problem
(Bradley & Goyal, 2008)
Lateral intraparietal (LIP) areaintegrates sensory evidence over time until a threshold is reachedopponent inhibitionused to make perceptual decisions about direction of motion
Race modelFirst pool to reach the threshold wins the race (Shadlen & Newsome, 2001; Smith & Ratcliff, 2004)
Reaction TimeTime it takes the winning pool to reach criterion12/1/2015
39Spiking Layer of LIP Decision Neurons
LIPMT PDSInput
Total of 80 trials (fixed motion strength, ten repetitions per motion direction)12/1/2015
40Perceptual Decision-Making
SNN model:71,026 neurons133 million synapsesLearning rule:Ca-based STDP (Brader et al., 2007)Categorization:Accumulator model (Smith & Ratcliff, 2004)
Achieved 92% accuracyReproduced RT distributions from human psychophysicsDecember 1, 201541Handwritten Digit Classification