CONFIDENTIAL Project Big Fish: Acquisition Opportunity July 2006 CHANGED TO JULY.
Electrosensory data acquisition and signal processing strategies in electric fish
description
Transcript of Electrosensory data acquisition and signal processing strategies in electric fish
![Page 1: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/1.jpg)
Electrosensory data acquisition and signal processing strategies
in electric fish
Mark E. Nelson
Beckman InstituteUniv. of Illinois, Urbana-Champaign
![Page 2: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/2.jpg)
How Electric Fish Work
![Page 3: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/3.jpg)
Distribution of Electric Fish
Fish tank upstairs
blackghost
knifefishelephant-
nosefish
![Page 4: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/4.jpg)
Electric Organ Discharge (EOD) - Spatial
![Page 5: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/5.jpg)
EOD - Temporal
![Page 6: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/6.jpg)
Electric Organ Discharge (EOD)
![Page 7: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/7.jpg)
Principle of active electrolocation
![Page 8: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/8.jpg)
mec
hano
MacIver, fromCarr et al., 1982
Electroreceptors
~15,000 tuberous electroreceptor organs1 nerve fiber per electroreceptor organ
up to 1000 spikes/s per nerve fiber
![Page 9: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/9.jpg)
Individual Sensors (Electroreceptors)
VIN
nerve spikesOUT
![Page 10: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/10.jpg)
Neural coding inelectrosensory afferent fibers
![Page 11: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/11.jpg)
Probability coding(P-type) afferent spike trains
00010101100101010011001010000101001010
Phead = 0.333
Phead = 0.337 Phead =
0.333
![Page 12: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/12.jpg)
Principle of active electrolocation
![Page 13: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/13.jpg)
Electrosensory Image Formation
![Page 14: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/14.jpg)
Electrosensory Image Formation
![Page 15: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/15.jpg)
Electrosensory Image Formation
![Page 16: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/16.jpg)
Prey-capture video analysis
![Page 17: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/17.jpg)
Prey capture behavior
![Page 18: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/18.jpg)
Fish Body Model
![Page 19: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/19.jpg)
Motion capture softwareMotion capturesoftware
![Page 20: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/20.jpg)
MOVIE: prey capture behavior
![Page 21: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/21.jpg)
Electrosensory Image Reconstruction
![Page 22: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/22.jpg)
Voltage perturbation at skin :Estimating Daphnia signal strength
waterprey
waterpreyfish arrE
/21/13
3
electrical contrastprey volume
fish E-field at prey
distance from prey to receptor
THIS FORMULA CAN BE USED TO COMPUTE THE SIGNAL AT EVERY POINT ON THE BODY
SURFACE
![Page 23: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/23.jpg)
MOVIE: Electrosensory Images
![Page 24: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/24.jpg)
System Capabilities
Electric fish can analyze electrosensory images to extract information on target
direction (bearing) distance size shape composition (impedance)
![Page 25: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/25.jpg)
Distance Discrimination
![Page 26: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/26.jpg)
Distance Discrimination
![Page 27: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/27.jpg)
Shape Discrimination
![Page 28: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/28.jpg)
Shape Discrimination
![Page 29: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/29.jpg)
Shape Generalization
![Page 30: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/30.jpg)
Shape “completion”
![Page 31: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/31.jpg)
Impedance Discrimination
![Page 32: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/32.jpg)
How Do They Do It? Electric fish analyze dynamic 2D electrosensory images on the body surface to determine target direction, distance, size, shape and
composition (impedance) Fish might perform an inverse mapping from 2D sensor data to obtain a dense 3D neural representation of world conductivity sensor data 3D conductivity action Alternatively, fish might use sensor data to directly estimate target parameters sensor data target parameters action
![Page 33: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/33.jpg)
Parameter estimation
(bearing)
![Page 34: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/34.jpg)
Parameter Estimation (cont.)
![Page 35: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/35.jpg)
Dynamic Movement Strategies
Fish are constantly in motion not a single, static ‘snapshot’ dynamic, spatiotemporal data stream
With respect to target objects in the environment, fish body movements simultaneously influence the relative positioning of the sensor array the electric organ effector organs (e.g. mouth)
![Page 36: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/36.jpg)
MOVIE: Electrosensory Images
![Page 37: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/37.jpg)
Active motor strategies: Dorsal roll toward prey
![Page 38: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/38.jpg)
Probing Motor Acts
chin probing back-and-forth (va et vient )
lateral probing
tangentialprobing
stationaryprobing
![Page 39: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/39.jpg)
Fish exploring a 4 cm cube
![Page 40: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/40.jpg)
CNS Signal Processing Strategies
Multi-scale filtering spatial and temporal
Adaptive background subtraction tail-bend suppression
Attentional ‘spotlight’ mechanisms local gain control
![Page 41: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/41.jpg)
Multiple Maps
![Page 42: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/42.jpg)
Multi-scale Filtering
INPUT(from skin receptors)
Centromedial map High spatial acuity Low temporal acuityCentrolateral map Inter spatial acuityInter temporal acuityLateral map Low spatial acuityHigh temporal acuity
tempo
ral
integ
ratio
n
bothspatial
integration
HINDBRAIN PROCESSING
PERIPHERALSENSORS
![Page 43: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/43.jpg)
Adaptive Background Subtraction
![Page 44: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/44.jpg)
Adaptive Background Subtraction
![Page 45: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/45.jpg)
Attentional ‘spotlight’ mechanism
![Page 46: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/46.jpg)
Summary Fish can evaluate direction, distance, size, shape and composition of target objectsHow? model-based parameter estimation based on 2D image
analysis, not full 3D reconstruction presumably some sort of (adaptive) (extended)
(unscented) Kalman-like algorithm extensive pre-filtering (virtual sensors?)
self-calibrating, adaptive noise suppression, multi-scale spatial and temporal signal averaging
dynamic control of source and array position
![Page 47: Electrosensory data acquisition and signal processing strategies in electric fish](https://reader035.fdocuments.us/reader035/viewer/2022070423/568167f1550346895ddd6425/html5/thumbnails/47.jpg)
Acknowledgements
Colleagues Curtis Bell (OHSU) Len Maler (Univ. Ottawa) Gerhard von der Emde (Univ. Bonn)
Nelson Lab Members Ling Chen, Rüdiger Krahe, Malcolm MacIver
Funding Agencies NIMH, NSF