Artificial retina : the multichannel processing of the ... · retina. We fitted our model to...

14
TB, DP, US, JNE/441349, 8/10/2012 IOP PUBLISHING JOURNAL OF NEURAL ENGINEERING J. Neural Eng. 9 (2012) 000000 (13pp) UNCORRECTED PROOF Artificial retina : the multichannel processing of the mammalian retina achieved with a neuromorphic asynchronous light acquisition device Henri Lorach 1,2,3 , Ryad Benosman 1,2,3 , Olivier Marre 1,2,3 , Sio-Hoi Ieng 1,2,3 , Jos´ e A Sahel 1,2,3,4,5,6 and Serge Picaud 1,2,3,6, 7 1 INSERM, U968, Institut de la Vision, 17 rue Moreau, Paris, F-75012, France 2 UPMC Univ Paris 06, UMR_S968, Institut de la Vision, 17 rue Moreau, Paris, F-75012, France 3 CNRS, UMR 7210, Institut de la Vision, 17 rue Moreau, Paris, F-75012, France 4 Centre Hospitalier National d’Ophtalmologie des Quinze-Vingts, 28 rue Charenton, Paris, F-75012, France 5 Institute of Ophthalmology, University College of London, London, UK 6 Fondation Ophtalmologique Adolphe de Rothschild, 25 rue Manin, Paris, F-75019, France E-mail: [email protected] Received 1 August 2012 Accepted for publication 25 September 2012 Published DD MM 2012 Online at stacks.iop.org/JNE/9/000000 Abstract Objective. Accurate modeling of retinal information processing remains a major challenge in retinal physiology with applications in visual rehabilitation and prosthetics. Most of the current artificial retinas are fed with static frame-based information, loosing thereby the fundamental asynchronous features of biological vision. The objective of this work is to reproduce the spatial and temporal properties of the majority of ganglion cell (GC) types in the mammalian retina. Approach. Here, we combined an asynchronous event-based light sensor with a model pulling nonlinear subunits to reproduce the parallel filtering and temporal coding occurring in the retina. We fitted our model to physiological data and were able to reconstruct the spatio-temporal responses of the majority of GC types previously described in the mammalian retina (Roska et al 2006 J. Neurophysiol. 95 3810–22). Main results. Fitting of the temporal and spatial components of the response was achieved with high coefficients of determination (median R 2 = 0.972 and R 2 = 0.903, respectively). Our model provides an accurate temporal precision with a reliability of only few milliseconds—peak of the distribution at 5 ms—similar to biological retinas (Berry et al 1997 Proc. Natl Acad. Sci. USA 94 5411–16; Gollisch and Meister 2008 Science 319 1108–11). The spiking statistics of the model also followed physiological measurements (Fano factor : 0.331). Significance. This new asynchronous retinal model therefore opens new perspectives in the development of artificial visual systems and visual prosthetic devices. Q1 (Some figures may appear in colour only in the online journal) 7 Author to whom any correspondence should be addressed. 1741-2560/12/000000+13$33.00 1 © 2012 IOP Publishing Ltd Printed in the UK & the USA

Transcript of Artificial retina : the multichannel processing of the ... · retina. We fitted our model to...

  • TB, DP, US, JNE/441349, 8/10/2012

    IOP PUBLISHING JOURNAL OF NEURAL ENGINEERING

    J. Neural Eng. 9 (2012) 000000 (13pp) UNCORRECTED PROOF

    Artificial retina : the multichannelprocessing of the mammalian retinaachieved with a neuromorphicasynchronous light acquisition deviceHenri Lorach1,2,3, Ryad Benosman1,2,3, Olivier Marre1,2,3,Sio-Hoi Ieng1,2,3, José A Sahel1,2,3,4,5,6 and Serge Picaud1,2,3,6,7

    1 INSERM, U968, Institut de la Vision, 17 rue Moreau, Paris, F-75012, France2 UPMC Univ Paris 06, UMR_S968, Institut de la Vision, 17 rue Moreau, Paris, F-75012, France3 CNRS, UMR 7210, Institut de la Vision, 17 rue Moreau, Paris, F-75012, France4 Centre Hospitalier National d’Ophtalmologie des Quinze-Vingts, 28 rue Charenton, Paris, F-75012,France5 Institute of Ophthalmology, University College of London, London, UK6 Fondation Ophtalmologique Adolphe de Rothschild, 25 rue Manin, Paris, F-75019, France

    E-mail: [email protected]

    Received 1 August 2012Accepted for publication 25 September 2012Published DD MM 2012Online at stacks.iop.org/JNE/9/000000

    AbstractObjective. Accurate modeling of retinal information processing remains a major challenge inretinal physiology with applications in visual rehabilitation and prosthetics. Most of thecurrent artificial retinas are fed with static frame-based information, loosing thereby thefundamental asynchronous features of biological vision. The objective of this work is toreproduce the spatial and temporal properties of the majority of ganglion cell (GC) types in themammalian retina.Approach. Here, we combined an asynchronous event-based light sensor with a model pullingnonlinear subunits to reproduce the parallel filtering and temporal coding occurring in theretina. We fitted our model to physiological data and were able to reconstruct thespatio-temporal responses of the majority of GC types previously described in the mammalianretina (Roska et al 2006 J. Neurophysiol. 95 3810–22).Main results. Fitting of the temporal and spatial components of the response was achievedwith high coefficients of determination (median R2 = 0.972 and R2 = 0.903, respectively).Our model provides an accurate temporal precision with a reliability of only fewmilliseconds—peak of the distribution at 5 ms—similar to biological retinas (Berry et al 1997Proc. Natl Acad. Sci. USA 94 5411–16; Gollisch and Meister 2008 Science 319 1108–11). Thespiking statistics of the model also followed physiological measurements (Fano factor : 0.331).Significance. This new asynchronous retinal model therefore opens new perspectives in thedevelopment of artificial visual systems and visual prosthetic devices.

    Q1 (Some figures may appear in colour only in the online journal)

    7 Author to whom any correspondence should be addressed.

    1741-2560/12/000000+13$33.00 1 © 2012 IOP Publishing Ltd Printed in the UK & the USA

    mailto:[email protected]://stacks.iop.org/JNE/9/000000

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    Introduction

    The mammalian retina samples and compresses visualinformation before sending it to the brain through the opticnerve. Complex features are already extracted by the retinalneural network such as movements in horizontal and verticaldirections by direction selective cells and even expandingpatterns [4, 5]. These feature extractions are not only veryprecise in the spatial domain but also in the temporal domain.Indeed, light responses to random patterns were found tobe reproducible with a millisecond precision [2, 3]. Moreover,the ganglion cell (GC) response latencies can reach 30 ms in theprimate retina [6] allowing thereby fast behavioral responses.For instance, static visual stimuli were shown to allow motordecision and the resulting motor response as early as 160 ms inmonkeys and around 220 ms in human subjects [7–9]. Visualprosthetic strategies should match retinal processing speed andtemporal precision. While some 20 different types of retinalGCs have been morphologically identified in mammals [10],more than half were individually recorded [11] such that theresponses to a given visual stimulation could be reconstructedin different populations of retinal GCs [1].

    To further understand biological vision, variouscomputational models of the retina were developed. Somehave intended to implement each biophysical step fromphototransduction at the level of photoreceptors (PRs) tospike initiation in GCs [12, 13]. These models involve aseries of nonlinear differential equations ruling the biophysicalprocesses involved in retinal signaling. Although they reacha high level of detail, these models are not suited forlarge-scale simulations of retinal responses. Other modelsare more functional and combine a linear filtering of thelight signal followed by a static nonlinearity and spikegeneration models [3, 2, 14, 15]. The spike generationmechanism can be either deterministic or probabilistic[15–17]. These models are qualified as linear–nonlinear (LN)and are computationally inexpensive because they involvelinear convolution operations. However, they do not reproduceresponses to natural stimuli for the vast majority of GC types[5] because retinal GCs are often selective to multiple visualfeatures at the same time. For example, ON–OFF directionselective cells cannot be modeled by the LN approach becauseboth ON (light increase) and OFF (light decrease) informationare summed nonlinearly. Moreover, their implementation oftenrelies on visual scene acquisition based on conventionalsynchronous cameras [18, 19]—as found in retinal implantdevices [20].

    An alternative approach is offered by recent developmentsin the field of neuromorphic devices. The principle ofneuromorphic engineering is to build electronic circuits thatmimic a neural processing [21] such as visual corticalfunctions: orientation detection and saliency mapping [22].Synapses and electrical coupling are replaced by transistorsand cell membranes by capacitors. Neuromorphic retinas, inparticular, have been developed for mapping retinal networkin silico and were able to output four GC types—ON and OFFtransient and sustained cells [23, 24]. Their spatio-temporalfiltering properties as well as light adaptation and responses

    to temporal contrasts could be partially reproduced. However,they still lacked the temporal precision of biological GCs anddid not respond to temporal frequencies higher than 10 Hz asopposed to mammalian retinas. Moreover, this artificial retinadid not perform the complex feature extractions of the othercell types.

    The advantage in using neuromorphic technology isstraightforward. First, the entire computation is performed insitu and does not need any dedicated processor, meaning amore compact device. Secondly, the hardware implementationof processing makes it low power consuming. And lastly,visual information can be sampled and processed in parallelover the entire field of view, matching thereby the retinaltemporal resolution. However, implementing the entire retinalnetwork in silico would require the integration of some 50different cell types of interneuron and GCs [10]. In thiscontext, there is no existing acquisition device able to outputthe majority of GC responses with the necessary temporalprecision.

    Here, we have implemented an intermediate strategybased on an asynchronous dynamic vision sensor (DVS) [25]also called ‘silicon retina’ by the inventors. This sensor detectslocal temporal contrasts but does not introduce temporaland spatial integrations occurring in biological retinas. Itprovides information about local light increments (ON) anddecrements (OFF). The purpose of this work was to produce acomputational model accounting for these integrations therebymatching temporal and spatial properties of different retinalGC types of the mammalian retina described in the literature[1]. The novelty in this study is to use this new asynchronoussensor instead of a classic frame-based camera to feeda computational model of retinal information processingkeeping thereby the asynchronous properties of biologicalvision.

    Methods

    Event-based sensor

    The temporal contrast DVS [25] that will be used in the restof the paper is an array of 128 × 128 autonomous pixels, eachof them asynchronously generating ‘spike’ events that encoderelative changes in pixel illumination, discarding the absolutelight value. The DVS does not deliver classical image data inthe form of static pictures with absolute intensity information.Rather, the device senses the temporal contrast informationpresent in natural scenes at temporal resolutions much higherthan what can be achieved with most conventional sensors. Inaddition, temporal redundancy that is common to all frame-based image acquisition techniques is largely suppressed. Thedigital event addresses containing the position of the pixel andthe polarity of the events are asynchronously and losslesslytransmitted using a four-phase handshake. The event ek isdefined by its spatial position (xk, yk), polarity pk and timeof occurrence tk:

    ek = (xk, yk, pk, tk). (1)The analogue visual information is therefore encoded in therelative timing of the pulses. For a detailed description of the

    2

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    (b)(a)

    (c) (d)

    Figure 1. Principle and output signals from an asynchronous dynamic vision sensor (DVS). (a) Sampling of light intensity into positive andnegative events by an individual pixel. The logarithm of the intensity log(I) reaching the pixel (top trace) drives the pixel voltage. As soon asthe voltage is changed by ±θ , since the last event from this pixel, an event is emitted (middle) and the pixel becomes blind during a givenrefractory period. Events generated by the sensor convey the timing of the threshold crossing and the polarity of the event (bottom). Positiveevents are plotted in red, negative events in blue. (b) 3D plot of the recorded signal from the dynamic vision sensor viewing a translatingblack disc. The position (x, y) and time t of the events are plotted along with the plan of motion. Positive events appear in red and negativeevents in blue, together drawing a tubular surface. (c) Experimental design to assess the temporal resolution of the dynamic vision sensorwith respect to that of a classic frame-based camera. Both devices were placed in front of a blinking LED. From the DVS signal, theblinking duration could be measured as the time difference between first positive events and first negative events (τDVS). The durationmeasured by the camera was the time difference between the time of the frame with light on and first frame with light off τframe.(d) Performances of the sensors. The frame-based camera (red triangles) was not able to measure a blinking duration below 40 ms. It waslimited by the sampling period of 33 ms. This behavior explains the horizontal asymptote for short durations of τLED. For higher values,τframe follows linearly the blinking duration. In contrast, the DVS (black squares) can measure blinking durations as low as 2 ms and followsthe ideal linear behavior from 2 ms to 1 s.

    sensor, see [25]. Briefly, it generates positive and negativeevents whenever the light intensity reaches a positive ornegative threshold. This threshold is adaptive and follows alogarithmic function thus mimicking PRs adaptation. Thissampling strategy for each pixel is summarized in figure 1(a).The light intensity I reaching the pixel is scaled by a logfunction (top). Every time log(I) is increased by θ from the lastevent, a positive event is emitted (middle). When it decreasesby θ , a negative event is generated. A refractory period duringwhich the pixel is blind limits the transmission rate of thesensor. This period is of the order of tens of microseconds.Therefore, very fast temporal variations of the light inputcan be sampled accurately and particular attention had to bepaid to avoid flickering induced by artificial ac light. Some

    light sources are indeed driven by alternating current, thusgenerating artifactual events. Adjusting the biases of the DVScould remove this effect by increasing the refractory period ofthe pixels. However, it would lower the temporal reliability ofthe sensor. To avoid flickering effects, we only used continuousillumination conditions in our experiments.

    To give a graphic representation of the sensors output,a spacetime plot of the events can be drawn (figure 1(b)).Here, the stimulus consisted in a translating black disc. Movingedges generated either positive (red in figure 1(b)) or negative(blue in figure 1(b)) events with a sub-millisecond temporalresolution. Thermal noise inside the pixels generated diffuseevents occurring randomly over the array. In this work, we

    3

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    used the DVS (#SN128-V1-104) with a lens Computar (16 mm1:1.4) at nominal biases.

    Temporal precision assessment

    To assess the timing resolution of the DVS compared to aconventional camera, an LED was controlled with a micro-controller (Arduino UNO) to deliver light flashes of durationτLED ranging between 1 ms and 1 s; we examined the minimumtiming for detecting a flash with either a conventional 30 Hzcamera (Logitech R©C500) or the dynamic vision sensor. Fromthe DVS signal, the blinking duration could be measured asthe time difference between the first positive events and thefirst negative events (τDVS), while it was the time differencebetween the time of the frame with light on and the firstframe with light off (τframe) for the frame-based camera.

    Electrophysiological data

    Patch clamp recordings from [1] were used to build the retinalmodel. Briefly, inhibitory currents, excitatory currents andspiking activity were recorded in response to a 600 μm squareof light flashed for 1 s at different positions around the cell. Tendifferent cell types were described and characterized both onelectrophysiological responses and morphological parameters.Excitatory and inhibitory currents were recorded by holdingthe membrane potential at −60 and 0 mV, respectively. Thisprotocol allowed us to segregate GC responses into fourdifferent components: ON-driven excitation and inhibition andOFF-driven excitation and inhibition.

    Parameter fitting

    We fitted the spatial and temporal parameters by nonlinearleast-squares method (Matlab, Levenberg–Marquardt algo-rithm). In the model, spatial and temporal components of theresponse were separated and fitted independently. The tem-poral parameters were fitted on the average temporal profilealong the spatial dimension. The spatial parameters were fittedon the average spatial receptive field over the first 200 ms afterstimulus onset and offset. Alpha functions classically describeexcitatory post-synaptic currents. Here, the temporal profileswere fitted with a sum of two alpha-functions. The spatial com-ponent of the filter was fitted using a 3-Gaussian combination:

    hspatial(x, y) =n∑

    i=1αie

    − (x−x0i )

    2+(y−y0i )2r2i (2)

    htemporal(t) =m∑

    i=1βi

    (t − t0i

    )e

    −(t−t0i )τi (3)

    with n = 3 and m = 2, introducing 15 free parameters. Fourdifferent sets of parameters were fitted over the excitatorycurrents from ON and OFF for both inhibition and excitation.

    Iterative implementation

    Figure 2 presents the system diagram. The events fromthe camera encoded by their position, polarity and time of

    Figure 2. Implementation principle. The DVS generates events (x, y,p, t) encoded by their position, polarity and time of occurrence (1).These events are transmitted to a computer (2) through a USBconnection. The timing t of the incoming event is binned at amillisecond precision t′. The state of the current (I) of 128×128cells is updated every millisecond (3). The new events contributes to Q2the evolution of I by the addition of the spatial kernel (4). From thisupdated value of the state I, spikes are emitted according to athreshold mechanism (5). The resulting spikes are encoded by theirposition and their millisecond timing (x′, y′, t′).

    occurrence (x, y, p, t) are transmitted to the computer through aUSB connection. The events are contributing to the calculationof the current I(t) of the 128×128 cells. The timing of the event(t) is approximated at the millisecond precision by the time (t′).The matrix I is therefore calculated every millisecond and athreshold detection generates output spikes (x′, y′ t ′,).

    The alpha-functions that we chose to describe thetemporal evolution of the currents and the independence ofthe spatial and temporal components allowed us to compute theoutput of the filters iteratively. The filter output was calculatedwith a 1 ms time step, thus providing a millisecond timingresolution regardless of the event sequence. Let fi and gi bedefined as fi(t) = βit e−

    tτi and gi(t) = βi e−

    tτi and the event

    e = (x, y, p, t). The evolution of the alpha-function fi wascalculated as

    fi(t + dt) = dt.e−dtτi g(t) + e− dtτi f (t) (4)

    gi(t + dt) = e−dtτi gi(t) + hspatial,i. (5)

    The contribution of the new event e was introducedby adding the corresponding spatial kernel hspatial,i =αi e

    − (x−xk+1−x0i )

    2+(y−yk+1−y0i )2r2i to the function gi.

    Spike generation mechanism

    From this input signal, an output spike was generated as soon asthe difference between the excitatory and inhibitory currentscrossed a threshold k� : k ∈ Z. This threshold was set toobtain a maximum firing rate of 200 Hz in the modeled cells.

    Uniform light stimulation

    The uniform random light stimulus was designed using Matlab(Psychtoolbox) and presented on an LED backlit display (iMacscreen, maximum luminance: 300 cdm−2). The intensity waschosen from a Gaussian distribution with 10% variance atmean light intensity of the monitor (∼150 cdm−2). The refreshrate was 60 Hz and the stimulus duration was 5 s. The samestimulus was repeated 50 times over the entire field of the DVSsensor and these conditions did not saturate the sensor.

    4

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    The time constants of the LED screen pixels may have aninfluence on the timing reliability of the model. Therefore, weassessed the effect of rise-time and fall-time of the screen bycomputing the autocorrelation of the DVS events of a singlepixel. The autocorrelation of the positive events accountedfor the rise-time of the screen, whereas the autocorrelationbetween negative events reflected the fall-time of the screen. Inboth cases, the autocorrelation displayed a sharp peak around1.5 ms.

    Statistical analysis of spiking activity

    Spike time variability of the modeled cells was assessed as thestandard deviation of reliable spike timings occurring acrosstrials when presenting the same stimulus. The repetition ofthe same stimulus evoked firing periods separated from eachother by periods of inactivity clearly appearing as peaks in theperistimulus time histogram (PSTH). However, peaks in thePSTH can be either due to high firing rate in one trial, or time-locked firing in many trials. We can discriminate these twosituations by quantifying the variability of the spike rate acrosstrials. If this variability exceeds a given threshold, it means thatthe firing period is not reliable. To estimate the reliability of theresponses, we applied the technique used in [2] to analyzebiological recordings. First, the timescale of modulation ofthe firing rate of a cell was assessed by cross-correlation ofspiking activity across different trials and fitting this cross-correlation with a Gaussian function (variance usually around20 ms). We used a smaller time bin (5 ms) to compute the PSTHallowing us to discriminate distinct periods of activity. Fromthe PSTH, reliable activity period boundaries were defined ifthe minimum ν between two surrounding maxima (m1, m2)was lower than

    √m1m21.5 with 95% confidence. This criterion

    rejects peaks in the PSTH that would be due to a few burstingcells only.

    For each of these reliable firing periods, the standarddeviation of the first spike time across trials was computed toassess timing reliability. This standard deviation was computedwithout binning, allowing us thereby to reach variabilitieslower than the 5 ms time bin. The variability in the number ofspikes was assessed by computing its variance in 20 ms timebins across trials. The Fano factor was calculated as the slopeof the linear fitting of the number of spikes per time bin againstits variance.

    Results

    Temporal precision of the sensor

    Spike timing is critical in retinal coding [2, 3, 26]. We thereforestarted by checking the timing accuracy of our light acquisitionsystem. If a millisecond precision is to be achieved in retinalcoding, our light sensor must at least reach this precision.To quantify this accuracy, we assessed the minimal durationof a blinking LED that could be measured with the DVS.Figure 1(c) shows how the dynamic vision sensor performs ascompared to a conventional frame-based camera. As expected,the conventional camera (figure 1(d) red triangles) was limitedby the sampling period of 30 Hz. It was not able to measure

    a blinking duration below these 33 ms corresponding to theacquisition of two successive frames. This behavior explainsthe horizontal asymptote for short LED blinking durations(below 33 ms), whereas the curve τframe follows linearly theblinking duration for higher values. In contrast, the DVS (blacksquares) can measure blinking durations as low as 2 ms withgreat reliability. The DVS behavior was linear from 2 ms to 1 s.This DVS sensor can thus detect changes in light intensitieswith a higher temporal resolution than conventional cameras.The ability of the sensor to follow high-frequency stimuliand increase temporal precision depended on illuminationconditions and bias settings. Increasing light intensity reducedboth event latencies and jitter down to 15μs ± 3.3% [25].Although mammalian cone PRs do not emit spikes, theycontinuously release neurotransmitter and respond over sevenlog-units of background illumination [27] and providing threelog-units dynamic range for a given background illumination.This adaptive feature is shared by the DVS. Moreover, conePRs display fast light responses. Although the PR peakphotocurrent is reached after 30 ms, a few picoampere changecan be obtained a few milliseconds only after stimulationonset [28, 29]. Such small current changes were shown tobe sufficient to change the cell potential and generate spike inretinal GCs [30]. These results are consistent with the 20 msresponse latency of the fastest retinal GCs [31]. Therefore, thedynamic vision sensor with its low latency appears sufficientto match the PR response kinetics to model the retinal GCresponses.

    Structure of the retinal model

    To determine if the DVS sensor could provide a usefulvisual information for a retinal model, we implementeda computational model of retinal information processing.At the retinal output, each GC type extracts a particularfeature of visual information based on its underlying network.Figure 3(a) illustrates the general structure of the neuralnetwork allowing this information processing although therespective operation by each neuronal step is not identicalfor all GC types. Phototransduction is performed by PRswith an adaptive behavior. They contact horizontal cells (HC)that provide lateral feedback inhibition. Bipolar cells (BC)are either excited (OFF-BC) or inhibited (ON-BC) by PRsglutamate release. In turn, they excite amacrine cells (AC) andGCs. AC can potentially receive inputs from both ON and OFFBC and provide inhibitory inputs to GCs. The different GCtypes behave as threshold detectors integrating inputs from BCand AC generating action potentials as soon as their membranepotential reaches a particular threshold.

    The DVS events may appear similar to the spikingactivity of transient retinal GCs. However, they lack thebiological temporal and spatial integration properties, such assurround inhibition. Inspired by this biological architecture,we implemented a model similar to [32, 33], segregating ONand OFF inputs to different channels and applying differentfilters to these components. This phenomenological modeldoes not reproduce the behavior of each interneuron but theirglobal equivalent inputs to the GCs. We applied temporaland spatial filtering stages to the DVS events (figure 3(b)).

    5

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    PR

    HC

    ON-BC OFF-BC

    GC

    AC

    excitationinhibition Θ

    (a) (b)

    Figure 3. Architecture of the computational model. (a) Architecture of the biological retina. (b) The different filtering stages of the model.ON events from the sensor provide inputs to ON bipolar cells and ON-driven amacrine cells and OFF events provide inputs to OFF bipolarcells and OFF-driven amacrine cells. These four filtering stages are further combined by the ganglion cell that emits a spike whenever thisinput signal varies from a threshold �.

    ON and OFF events ek = (xk, yk, pk, tk) were segregated andconvolved with different filters (see section ‘Methods’), passedthrough a static nonlinearity (N) and summed to provide GCinput current. An output spike was emitted as soon as the inputcurrent I varied up to a threshold �. This threshold was chosento obtain a maximum output firing rate of 200 Hz. The modelwas implemented in Matlab as follows:

    I(x, y, t) = IexcON − IinhON + IexcOFF − IinhOFF (6)

    Iexc,inhON,OFF(x, y, t)

    = N⎛⎝∑

    k:tk

  • J.NeuralE

    ng.9(2012)

    000000H

    Lorach

    etal

    Table 1. Parameters list. Each current (ON-excitation, ON-inhibition, OFF-excitation, OFF-inhibition) is described by a set of 15 parameters including 6 parameters for the temporal componentand 9 parameters for the spatial profiles. LED: local edge detector.

    α1 t01 τ1 α2 t02 τ2 β1 r1 x

    01 β2 r2 x

    02 β3 r3 x

    03

    (pA ms−1) (ms) (ms) (pA ms−1) (ms) (ms) (pA) (μm) (μm) (pA) (μm) (μm) (pA) (μm) (μm)

    ON BETA ONexc 292 76.7 44.6 72.4 8.03e−07 277 −395 74.3 74.4 398 75.6 71.7 125 2.27 52.4ONinh 87.8 69.2 63.5 130 0.809 608 1.59 371 41.4 −1.41 97.6 180 839 1.56 2.9OFFexc 43.3 147 63.8 43.2 147 63.8 7.07 324 −4.82 −6.75 302 −5.61 −2.33 46.3 17.3OFFinh 41.4 85.5 26 79.6 112 25.1 6.1 87 −0.0886 31.5 249 −9.78 −34.1 234 −7.81

    OFF BETA ONexc 141 275 52.9 89.5 208 59 10.4 22.5 146 −19.5 0.698 72.3 5.5 75.2 −170ONinh 23.9 93.1 56 23.9 93 56 −2.6 218 169 3.69 449 −42.7 −2.91 277 −200OFFexc 168 94 168 57.1 61.7 1e+03 18.7 55.1 134 −69.8 106 57 76.9 84.8 39.9OFFinh 31.7 67.3 46.5 13.9 83.2 390 0.382 950 −44.1 −0.721 12 96.4 −2.34 35.6 34.4

    ON-PARASOL ONexc 104 89.9 28 242 62.3 122 −7.41 25 44.3 23.9 170 114 −23.3 122 161ONinh 142 71.1 46.9 15.5 7.2e−06 403 −2.9 211 −22.5 1.96 895 −85.2 5.27e+03 0.823 52.1OFFexc 30.5 92.9 1e+03 30.5 92.9 1e+03 3.76 271 18.9 −6.67 106 −7.85 −2.48 70.2 184OFFinh 81.1 106 81.2 64.1 184 23.2 −222 49 39.3 225 49.5 38.8 0.594 1e+03 −174

    OFF PARASOL ONexc 30.7 88 36 17.6 67.1 167 −1.23 43.3 115 −2.57 165 −16.5 1.7 395 −13.2ONinh 14.8 4.8e−05 604 66.4 69.1 61 0.936 104 −38.5 −8.89 1.56 83.4 0.467 511 −200OFFexc 96.9 72.5 41.2 32.3 42.1 223 1.55 214 87.2 −14.6 92.9 168 13.3 88 168OFFinh 39.7 65.3 45.5 37.6 108 41 1.67 74.8 188 −3.84 693 178 3.47 1e+03 170

    ON DELTA ONexc 58.4 44.1 723 91.5 57.9 148 −1.63 38.8 −15.4 2 19.8 93.4 2.19 227 −4.77ONinh 423 64.8 61.7 11 6.54 309 3.13 589 30.8 8.9 82.7 38.2 −15.5 29.6 94OFFexc 36.7 13.8 2.21 25.4 16 2.08 −3.17 301 −7.11 1.16 378 −21.2 2.23 247 −0.717OFFinh 324 83.6 137 59.9 176 1e+03 −7.62 240 24.3 9.15 447 7.9 −5.57 62.8 78.3

    OFF DELTA ONexc 74.2 76.2 97.5 27.4 20.6 386 −139 0.746 91 −4.87 31.8 −25.6 1.38 443 81.8ONinh 66.4 30.8 340 114 102 51.4 2.02 582 −163 1.36 136 200 −13.5 49.2 43.2OFFexc 44.6 72.3 156 44.6 72.3 156 76 28.2 21.7 −76.3 28.5 21.5 0.645 438 81OFFinh 21.4 1.15 32.5 113 116 32 2.68 658 −200 −3.36 465 −104 4.79 97.9 18.3

    ON BISTRAT ONexc 556 68.8 52.7 40.4 7.32e−06 935 71.7 92.7 12 −58.5 95.7 17.6 3.51 343 20.5ONinh 71.2 106 213 67.8 82.4 37 0.929 966 45.5 −3.87 43.4 174 −1.4 193 −113OFFexc 71 211 30 52.7 145 29.6 −7.07 207 45.8 5.93 269 45 0.82 24.5 144OFFinh 77.2 94.9 51 64.9 126 245 3.22 16.4 131 2.56 64.5 23.1 0.276 1e+03 139

    OFF COUPLED ONexc 8.46 154 70.2 6.05 146 369 0.331 13 36.9 0.0903 845 −62.7 −0.276 125 −17.2ONinh 45 89.5 47.4 39.6 33.9 590 −0.958 520 168 1.09 829 200 0.872 136 −24.1OFFexc 5.81 61.6 12.5 7.33 80.5 8.82 50.8 58.9 30.7 1.38 78.9 24.9 −52 59.5 30.6OFFinh 27.2 1.14 2.86 0.002 45 237 3.33 0.122 347 −200 0.163 1e+03 183 −0.404 106 22

    LED ONexc 29.4 58.4 299 48.6 90.2 105 −33.4 266 36.8 −3.63 31.4 35.4 33.9 274 37.1ONinh 124 71.4 139 87.1 132 21.9 3.03 299 −200 4.23 244 168 −6.01 135 51.7OFFexc 21.1 72.5 123 21.1 72.5 123 22.2 188 14.3 −42.6 212 17.4 21 236 20.4OFFinh 130 99 29.1 82.7 113 122 26 334 −53.1 5.55 55.6 −12.8 −25.7 320 −52.4

    ON−OFF DS ONexc 16.5 4.92e−06 561 73.7 123 128 −4.09 56.7 1.17 43.8 361 175 −42.7 354 183ONinh 18.9 94.9 22.3 10.6 54.2 29.4 5.04 47.8 −9.85 16.8 74.1 76.4 −16.2 89.7 65OFFexc 0.000 233 0.000 739 13.4 0.000 233 0.000 739 13.4 0.431 39.7 107 −0.487 92.7 47.6 0.123 491 −24.4OFFinh 32.4 100 44.7 31.9 148 514 −3.45 90.8 −127 23 240 86.8 −23 220 102

    7

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    Figure 4. Reconstruction of an ON-Beta ganglion cell excitatory currents in response of a square of light. Experimental data from [1] wereobtained by recording excitatory currents (holding the membrane potential at −60 mV) in response to a square of light flashed at differentpositions. The stimulus consisted in a 600 μm square of light flashed for 1s across the ganglion cell with 60 μm steps. Each row of the colorplot represents the temporal response current at the respective position of the light square. Model parameters were fitted on the spatial (top)and temporal (bottom) averages of the currents for both ON phase and OFF phase.

    threshold �. This threshold was set to achieve a maximumfiring rate of 200 Hz in the modeled cells.

    Spike timing and count variability

    Variability in GC responses has been characterized in vitroand is thought to play an important role in retinal coding [2].In order to quantify the variability in our artificial retina, weused a random stimulation protocol. Spatially uniform randomlight stimulations are often used to probe retinal function[2, 26, 36]. Here, we assessed the bursting behavior of themodeled cells. Figure 7(a) presents the reliable activity of theGCs. The stimulus consisted in a spatially uniform Gaussiandistributed intensity with 10% variance in contrast refreshedat 60 Hz. It evoked typical bursting activity shown by theraster plots for 50 repetitions of the same stimulus. Each cellresponds with a particular temporal signature in its activityperiods underlining the importance of temporal coding.

    The timing reliability of the modeled cells was analyzed aspreviously described in [2]. Spike time variability was assessed

    as the standard deviation of the first spike in an activitywindow occurring across trials when presenting the samestimulus. Indeed, the repetition of the same stimulus evokedfiring events separated from each other by periods of inactivity(figure 7(a)). Although these activity periods occur clearly asvertical bands in the raster plots, we provide a mathematicalcriterion for separating spiking periods from one another (seesection ‘Methods’). In trials where no spike appeared withinan activity window, no contribution was made to the standarddeviation calculation [2]. The variability was calculated as thestandard deviation of the timing of the first spike in the activityperiod across trials. Similar measures have been often used andallow comparisons with existing data. The timing variabilityhistogram for all cells is presented in figure 7(b). Some cellsdisplayed timing reliability as low as few milliseconds witha maximum value of the histogram at 5 ms. This result isin agreement with physiological findings in mammalian GCs[2, 26].

    The variability in the number of spikes across trialswas illustrated by expressing the mean number of spikes

    8

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    Figure 5. Fitting of temporal parameters. Temporal response profiles were split into four different components. The ON-driven excitationand inhibition corresponding to the stimulus onset and the OFF-driven excitation and inhibition generated by the stimulus offset. Recorded

    data (black curves) were fitted by a sum of two alpha-functions (red curves)∑2

    i=1 βi(t − t0i ) exp( −(t−t0i )

    τi) and provided good coefficient of

    determination for all cells (median R2 = 0.972).

    in a 20 ms time bin against its variance over the 50 trials(figure 7(c)). As expected, this spiking variability was muchlower than predicted from Poisson statistics (following thelinear relationship plotted). The linear fitting of the mean-to-variance relationship gave a Fano factor of 0.331, to becompared to the 0.277 found in primate GCs [26]. Thislow variability means that a significant amount of visualinformation is represented by the number of spikes at the 20 mstimescale [35].

    Discussion

    Taking advantage of a fast asynchronous sensor with a widedynamic range, we showed that our artificial retina couldreproduce the temporal precision of the mammalian retinafor different visual information channels. The main advantagerelies in the high temporal resolution of the sensor andthe event-based representation of the visual information. Thedynamic vision sensor mimics the adaptive and asynchronouscharacteristics of phototransduction. Its wide dynamic range(120 dB) quantitatively matches the sensitivity of the humanretina. It is able to respond to illuminance ranging from 1 to105 lux [25]. By responding to logarithmic temporal contrasts,it provides a compact signal that can be further processed to

    mimic retinal processing. Despite the transient nature of theDVS signal, both transient and more sustained cells could bemodeled by introducing different time constants in the alpha-functions—ranging from 22 ms to 1 s (see table 1). However,longer time constant or dc components of the light signal werenot taken into account.

    Moreover, the event-based nature of the DVS does notreproduce the analogue nature of the PR signal. The conversionfrom spikes to analogue signal and to spikes back again in ourimplementation may seem puzzling and we agree that it doesnot faithfully reproduce retinal physiology itself. The retinaindeed remains analogous until spike generation in GCs—and in some AC. However, the asynchronous nature of theDVS signal and its low latency are critical advantages forthe temporal precision of the model. Moreover, the generalnature of the DVS output signal allows flexible processing toreproduce multiple different cell types from the same signal.

    Conventional retinal models include linear ones wherespike rate is linearly dependent on light intensity, LN,stochastic or not [15, 17] and integrative models where eachion channel of each cell is accounted for [12]. However,practical use of these models always relies on an acquisitiondevice. For the purpose of implementing an artificial retina,a millisecond timing precision seems critical. Therefore, the

    9

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    Figure 6. Fitting of spatial parameters. Spatial response profiles were split into the four components: ON-driven excitation and inhibitionand OFF-driven excitation and inhibition. The recorded spatial profiles (black curves) were fitted using a sum of three Gaussian functions∑3

    i=1 αi exp(− (x−x0i )

    2

    r2i) (red curves). This fitting was performed by nonlinear least-squares method and provided a good coefficient of

    determination for the vast majority of cells (median R2 = 0.903).

    accuracy of the sensor itself is an important limiting factor.Here, we circumvented this problem by using the dynamicvision sensor and were able to match the millisecond timingaccuracy of retinal coding. We showed that the use of anasynchronous neuromorphic sensor is well suited for artificialretina implementation.

    We based our work on physiological experiments in theliterature. We fitted our parameters on responses from rabbitGCs obtained with a relatively simple stimulus. Therefore wedid not account here for more complex behaviors. However,we provided evidence that timing precision and reliability aswell as spiking statistics in response to a flickering randomstimulus were achieved by our artificial retina. A parameterfitting on stimuli with natural statistics would improve theaccuracy of the model and would unravel more complexbehaviors. Moreover, multiple sources of noise such as photonnoise, synaptic noise and channel noise are not included in ourimplementation in which noise comes from thermal noise inthe pixel and photon absorption noise.

    In theory, the DVS and AER are not essential to reproducethe GC responses and their temporal precision. This can indeedbe done by a simple model like, for example, the one in[36]. However, implementing this model in a device raisesseveral challenges, to which the DVS provides an elegant

    solution. Alternatively, reaching a similar precision with theframe-based camera would require a high computational cost.As pointed out earlier, high frame rate cameras alreadyreaching the kHz-sampling frequency are usually well suitedfor slow motion analysis in industry and research and wouldtheoretically be fast enough to compute accurate spike trains.However, these devices are power consuming and inefficient.First, they produce data regardless of the changes in a sceneso that a 1000 × 1000 pixel array at 1 kHz with a 12-bitsprecision transfers 1.5 GB per second. Moreover, due to theirshort exposure times, they need high illumination levels as wellas heavy cooling system to reduce thermal noise and improvesignal to noise ratio. For all these reasons, fast cameras usingframes are expensive, heavy and large, high energy consumingand thus not suited for embedded applications where lightconditions are not necessarily controlled and where efficiencyin power and data production is a key challenge. Asynchronoussensors offer promising results for these kinds of applications.As opposed to the frame-based cameras, data are producedexclusively when motion occurs, thus compressing visualinformation prior to any processing. Moreover, the continuousexposure of the pixels and the adaptive threshold provide ahigh sensitivity in a wide dynamic range (120 dB) withoutany cooling system. Finally, the event-based nature of the

    10

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    (a) (b)

    (c)

    Figure 7. Response reproducibility in the modeled retinal ganglion cell. (a) Cell responses to a spatially uniform random flicker. Thestimulus consisted in a spatially uniform random illumination (red trace) from a Gaussian distribution with 10% variance updated at 60 Hz.Raster plots of the cell response to the 50 consecutive trials are represented. Reliable firing events for individual cells appear as vertical barsin the raster plot. Each cell display a specific time coding for the same stimulus. (b) Histogram of the spike timing variability. The variabilityis computed for each reliable spiking period as the standard deviation in the millisecond timing of the first spike across trials (taken withoutbinning). This variability can be as low as 2 ms for some spiking periods. The histogram maximum for all cells is 5 ms in good agreementwith biological results [2]. (c) Spike count variability. The mean number of spikes per event across trials is plotted against its variance. Thestraight line represents equality between mean and variance. This relationship is valid if the spiking probability follows a Poisson lawdepending on the stimulus strength. Here, the variability is lower than expected from a Poisson process meaning that the number of spikesper reliable spiking periods is reproducible across trials. The Fano factor obtained by linear fitting of the mean/variance relationship (bluedotted line) is lower than unity (0.331).

    DVS signal would allow us to use event-based strategies forsimulating neural networks [37].

    Here, as a first step, we implemented the model in Matlabto fit the spatial and temporal parameters. Using computersimulations, we were not able to achieve real time simulationof the retinal GCs. The computational cost of the model scaleswith the number of events and the size of the spatial filter. Here,typical filter sizes were 10 × 10 allowing us to process around104 events per second with a conventional processor underMatlab. Natural scenes generate around 105 events per secondon the DVS; therefore our implementation method is 10 timesslower than real time. This lack of efficiency was mainly due tothe synchronization step when calculating the evolution of theinput currents iteratively at a millisecond resolution. However,once the model parameters are set, it is possible to increasecomputation speed using C, GPU or FPGA programming.Asynchronous convolution chips are also under development[38, 39] so that a hardware implementation of this retinal modelcould be performed. However, the long time constants present

    in our model—up to 1 s—would be difficult to implement inpurely analogue circuits because this would require very largecapacitances or very low currents, leading to prohibitivelylarge area and/or mismatch of the circuits. Mixed analogue-to-digital circuits could however be used to implement large timeconstants. Such an implementation would constitute a key stepto reach real time processing in embedded devices.

    Another application of such embedded vision device couldbe for medical purposes. Indeed, patients suffering from retinaldegeneration can lose part of their visual field. This lossthat may be either central or peripheral can be compensatedby visual aids for visually impaired patients providing themissing information in an augmented reality fashion. Anexternal camera is used to extract relevant information toprovide to the patient. Therefore, the use of an efficient sensorand a biomimetic model can greatly improve these kindsof embedded systems. In the case of complete blindness,functional vision rehabilitation can be achieved by retinalor cortical prostheses stimulating remaining retinal cells or

    11

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    Figure 8. Parallel coding of a complex stimulus. The person isturning the head toward the left of the image. The DVS imagerepresent the accumulation of positive and negative events generatedby the sensor over 20 ms. The next images represent the scaledcolormaps of input currents to different populations of retinalganglion cells allowing us to define visual information sent to thebrain by the different parallel channels during complex stimulations.

    directly cortical cells, respectively. This stimulation can beeither an electrical excitation of the neurons [40] or a light-driven stimulation toward genetically modified cells that wereartificially made light sensitive [41]. Two stimulation sites

    are considered for retinal prostheses, either at the level of thedegenerated PRs in the subretinal space or at an epiretinalposition to directly activate retinal GCs. In these epiretinalimplants as well as in cortical prostheses, an importantvisual information processing is required to provide adequatestimulations to the cells. Since single cell stimulation startsto emerge [41, 42], our model provides different informationchannels that could be sequentially tested in patients in order toselect the most efficient one because each information channelin the retina might be devoted to specific task. In the field ofvisual implants, the main goal for the patients is to achieveface recognition task or reading tasks. In this context, ourmodel can be used to evaluate which one of the differentinformation channels is more appropriate to achieve such tasks.Figure 8 illustrates the encoding of a moving face throughthe ten information channels underlining different featuresof the image. For cortical implants, our model needs to becompleted by higher order computational processes occurringbetween the retina and the targeted brain area in order toprovide time accurate and meaningful visual information. Inall cases, the asynchronous nature of the generated signal couldprovide information to the stimulating device, thus reducingthe power required for existing conventional image conversionsystems. This point will be critical in a near future as pixelnumbers is likely to increase exponentially requiring morepowerful image analyzers connected to conventional frame-based cameras. DVS sensors may thus reduce the need forpowerful image processors for retinal modeling and prostheticapplications.

    Acknowledgments

    We thank Dr Posch, Dr Frégnac and Dr Guiraud for helpfuldiscussions on the project. We also thank the reviewers for theirconstructive remarks. This work was supported by INSERM,Université Pierre et Marie Curie (Paris VI), FondationOphtalmologique A de Rothschild (Paris), Agence Nationalepour la Recherche (ANR RETINE), ITMO Technologie pourla santé, the Fédération des Aveugles de France, IRRP, the cityof Paris, the Regional Council of Ile-de-France. HL receiveda doctoral fellowship from the Ecole Polytechnique.Author contributions. All authors contributed extensively tothe work presented in this paper.

    References Q3

    [1] Roska B, Molnar A and Werblin F S 2006 Parallel processingin retinal ganglion cells: how integration of space-timepatterns of excitation and inhibition form the spiking outputJ. Neurophysiol. 95 3810–22

    [2] Berry M J, Warland D K and Meister M 1997 The structureand precision of retinal spike trains Proc. Natl Acad. Sci.USA 94 5411–6

    [3] Gollisch T and Meister M 2008 Rapid neural coding in theretina with relative spike latencies Science 319 1108–11

    [4] Munch T A, da Silveira R A, Siegert S, Viney T J,Awatramani G B and Roska B 2009 Approach sensitivity inthe retina processed by a multifunctional neural circuitNature Neurosci. 12 1308–16

    12

    http://dx.doi.org/10.1152/jn.00113.2006http://dx.doi.org/10.1073/pnas.94.10.5411http://dx.doi.org/10.1126/science.1149639http://dx.doi.org/10.1038/nn.2389

  • J. Neural Eng. 9 (2012) 000000 H Lorach et al

    [5] Gollisch T and Meister M 2010 Eye smarter than scientistsbelieved: neural computations in circuits of the retinaNeuron 65 150–64

    [6] Schmolesky M T, Wang Y, Hanes D P, Thompson K G,Leutgeb S, Schall J D and Leventhal A G 1998 Signaltiming across the macaque visual system J. Neurophysiol.79 3272–8

    [7] Thorpe S, Fize D and Marlot C 1996 Speed of processing inthe human visual system Nature 381 520–2

    [8] Fabre-Thorpe M, Richard G and Thorpe S J 1998 Rapidcategorization of natural images by rhesus monkeysNeuroReport 9 303–8

    [9] Van Rullen R and Thorpe S J 2011 Rate coding versustemporal order coding: what the retinal ganglion cells tellthe visual cortex Neural Comput. 13 1255–83

    [10] Masland R H 2001 The fundamental plan of the retina NatureNeurosci. 4 877–86

    [11] Roska B and Werblin F 2001 Vertical interactions across tenparallel, stacked representations in the mammalian retinaNature 410 583–7

    [12] Publio R, Oliveira R F and Roque A C 2009 A computationalstudy on the role of gap junctions and rod conductance inthe enhancement of the dynamic range of the retina PLoSONE 4 e6970

    [13] Balya D, Roska B, Roska T and Werblin F S 2002 A CNNframework for modeling parallel processing in amammalian retina Int. J. Circuit Theory Appl.30 363–93

    [14] Meister M, Michael J and Berry I 1999 The neural code of theretina Neuron 22 435–50

    [15] Pillow J W, Paninski L, Uzzell V J, Simoncelli E Pand Chichilnisky E J 2005 Prediction and decoding ofretinal ganglion cell responses with a probabilistic spikingmodel J. Neurosci. 25 11003–13

    [16] Wohrer A and Kornprobst P 2009 Virtual retina : a biologicalretina model and simulator, with contrast gain controlJ. Comput. Neurosci. 26 219–49(arXiv:10.1007/s10827-008-0108-4)

    [17] Pillow J W, Shlens J, Paninski L, Sher A, Litke A M,Chichilnisky E J and Simoncelli E P 2008 Spatio-temporalcorrelations and visual signalling in a complete neuronalpopulation Nature 454 995–9

    [18] Werblin F 1991 Synaptic connections, receptive fields andpatterns of activity in the tiger salamander retina. Asimulation of patterns of activity formed at each cellularlevel from photoreceptors to ganglion cells Invest.Ophthalmol. Vis. Sci. 32 2440 (the Friendenwald lecture)

    Werblin F 1991 Invest. Ophthalmol. Vis. Sci. 32 459–83(erratum)

    [19] Vörösházi Z, Nagy Z and Szolgay P 2009 FPGA-based realtime multichannel emulated-digital retina modelimplementation EURASIP J. Adv. Signal Process.2009 6:1–6:10

    [20] Humayun M S et al 2003 Visual perception in a blind subjectwith a chronic microelectronic retinal prosthesis Vis. Res.43 2573–81

    [21] Mead C 1989 Analog VLSI and Neural Systems (Boston, MA:Addison-Wesley)

    [22] Vogelstein R J, Mallik U, Culurciello E, Cauwenberghs Gand Etienne-Cummings R 2007 A multichip neuromorphicsystem for spike-based visual information processingQ4

    [23] Zaghloul K A and Boahen K A 2005 An ON–OFF log domaincircuit that recreates adaptive filtering in the retina IEEETrans. Circuits Syst. I 52 99–107

    [24] Zaghloul K A and Boahen K 2006 A silicon retina thatreproduces signals in the optic nerve J. Neural Eng.3 257–67

    [25] Lichtsteiner P, Posch C and Delbruck T 2008 A 128 x128-120db-15us latency asynchronous temporal contrastvision sensor IEEE J. Solid-State Circuits 43 566–76

    [26] Uzzell V J and Chichilnisky E J 2004 Precision of spike trainsin primate retinal ganglion cells J. Neurophysiol. 92 780–9

    [27] Perlman I and Normann R A 1998 Light adaptation andsensitivity controlling mechanisms in vertebratephotoreceptors Prog. Retin. Eye Res. 17 523–63

    [28] Schnapf J L, Nunn B J, Meister M and Baylor D A 1990Visual transduction in cones of the monkey Macacafascicularis J. Physiol. 427 681–713

    [29] Schneeweis D M and Schnapf J L 1999 The photovoltage ofmacaque cone photoreceptors: adaptation, noise andkinetics J. Neurosci. 19 1203–16

    [30] Baylor D A and Fettiplace R 1977 Transmission fromphotoreceptors to ganglion cells in turtle retina J. Physiol.271 391–424

    [31] Bolz J, Rosner G and Wässle H 1982 Response latency ofbrisk-sustained (x) and brisk-transient (y) cells in the catretina J. Physiol. 328 171–90 Q5

    [32] Gollisch T and Meister M 2008 Modeling convergent on andoff pathways in the early visual system Biol. Cybern.99 263–78

    [33] Fairhall A L, Burlingame C A, Narasimhan R, Harris R A,Puchalla J L and Berry M J 2006 Selectivity for multiplestimulus features in retinal ganglion cells J. Neurophysiol.96 2724–38

    [34] Nirenberg S, Carcieri S M, Jacobs A L and Latham P E 2001Retinal ganglion cells act largely as independent encodersNature 411 698–701

    [35] de Ruyter van Steveninck R R, Lewen G D, Strong S P,Koberle R and Bialek W 1997 Reproducibility andvariability in neural spike trains Science 275 1805–8

    [36] Berry I I, Michael J and Meister M 1998 Refractoriness andneural precision J. Neurosci. 18 2200–11

    [37] Rudolph-Lilith M, Dubois M and Destexhe A 2012 Analyticalintegrate-and-fire neuron models with conductance-baseddynamics and realistic postsynaptic potential time coursefor event-driven simulation strategies Neural Comput.24 1426–61

    [38] Serrano-Gotarredona R et al 2009 Caviar: a 45k neuron, 5Msynapse, 12G connects/s AER hardwaresensory-processing-learning-actuating system forhigh-speed visual object recognition and tracking IEEETrans. Neural Netw. 20 1417–38

    [39] Linares-Barranco A, Paz-Vicente R, Gó andmez Rodriguez F,Jimé andnez A, Rivas M, Jimé andnez G and Civit A 2010On the aer convolution processors for FPGA ISCAS: Proc.IEEE Int. Symp. on Circuits and Systems pp 4237–40

    [40] Zhou D D and Greenbaum E 2009 Implantable NeuralProstheses 1: Devices and Applications vol 1 (Biologicaland Medical Physics, Biomedical Engineering) (Berlin:Springer)

    [41] Anding B, Cui J, Yu-Ping M, Olshevskaya E, Mingliang P,Dizhoor A M and Pan Z-H 2006 Ectopic expression of amicrobial-type rhodopsin restores visual responses in micewith photoreceptor degeneration Neuron 50 23–33

    [42] Sekirnjak C, Hottowy P, Sher A, Dabrowski W, Litke A Mand Chichilnisky E J 2008 High-resolution electricalstimulation of primate retina for epiretinal implant designJ. Neurosci. 28 4446–56

    13

    http://dx.doi.org/10.1016/j.neuron.2009.12.009http://dx.doi.org/10.1038/381520a0http://dx.doi.org/10.1097/00001756-199801260-00023http://dx.doi.org/10.1162/08997660152002852http://dx.doi.org/10.1038/nn0901-877http://dx.doi.org/10.1038/35069068http://dx.doi.org/10.1371/journal.pone.0006970http://dx.doi.org/10.1002/cta.204http://dx.doi.org/10.1016/S0896-6273(00)80700-Xhttp://dx.doi.org/10.1523/JNEUROSCI.3305-05.2005http://dx.doi.org/10.1007/s10827-008-0108-4http://arxiv.org/abs/10.1007/s10827-008-0108-4http://dx.doi.org/10.1038/nature07140http://dx.doi.org/10.1016/S0042-6989(03)00457-7http://dx.doi.org/10.1007/978-1-4613-1639-8http://dx.doi.org/10.1109/TCSI.2004.840097http://dx.doi.org/10.1088/1741-2560/3/4/002http://dx.doi.org/10.1109/JSSC.2007.914337http://dx.doi.org/10.1152/jn.01171.2003http://dx.doi.org/10.1016/S1350-9462(98)00005-6http://dx.doi.org/10.1007/s00422-008-0252-yhttp://dx.doi.org/10.1152/jn.00995.2005http://dx.doi.org/10.1038/35079612http://dx.doi.org/10.1126/science.275.5307.1805http://dx.doi.org/10.1162/NECO_a_00278http://dx.doi.org/10.1016/j.neuron.2006.02.026http://dx.doi.org/10.1523/JNEUROSCI.5138-07.2008

  • QUERIES

    Page 1Q1Author: Please be aware that the color figures in this articlewill only appear in color in the Web version. If you requirecolor in the printed journal and have not previously arrangedit, please contact the Production Editor now.

    Page 4Q2Author: Please check whether the edits made in the sentence‘The state of the current (I) of...’ in caption to figure 2 retainyour intended sense.

    Page 12Q3Author: Please check the details for any journal references thatdo not have a blue link as they may contain some incorrectinformation. Pale purple links are used for references to arXive-prints.

    Page 13Q4Author: Please provide the complete bibliographic informationin reference [22].

    Q5Author: Please check whether the surname of the third authoris okay as amended in reference [31].

    IntroductionMethodsEvent-based sensorTemporal precision assessmentElectrophysiological dataParameter fittingIterative implementationSpike generation mechanismUniform light stimulationStatistical analysis of spiking activity

    ResultsTemporal precision of the sensorStructure of the retinal modelResponse to a flashed stimulusSpike timing and count variability

    DiscussionAcknowledgmentsReferences