Simplifying Neurorobtic Development with NCSTools

C. M. Thibeault1,2,3,*, J. Hegie2, L. Jayet Bray2,3, and F. C. Harris Jr.2,3

1Department of Electrical and Biomedical Engineering, University of Nevada, Reno. Reno, NV.2Department of Computer Science, University of Nevada, Reno. Reno, NV.

3Brain Computation Lab, University of Nevada, Reno. Reno, NV.

Abstract

The combination of biologically realistic neural simula-tions and robotic agents offers unique opportunities forboth computational neuroscience and research in intelli-gent robotics. A concept that can provide insights intothe cognitive developments involved in human roboticinteraction as well as provide a pathway to developingtruly intelligent agents. In this paradigm spiking neuralmodels are coupled with physical or virtual entities in aclosed-loop. The embodied agent provides stimulus tothe neural model which in turn provides a filtered andprocessed view of that world. More often than not thecomplexity and corresponding computational burden ofthese models necessitates the use of high-performancecomputers that must be stored and maintained separatefrom the entity. In these instances, closing the loop canbe an arduous task requiring intimate collaboration be-tween neuroscientists and engineers. Presented here is asoftware package for simplifying those interactions whilefacilitating the communication between neural simula-tions and abstract remote entities.

Keywords: Neurorobotics, Spiking Neural NetworkSimulation, Autonomous Agents, Intelligent Robotics.

1 Introduction

Computational neuroscience enjoys a unique role in bi-ological research. At one end it can help validate andquantify experimental results. While at the other, itprovides a predictive mechanism for aspects of the ner-vous system that are unreachable by any other means.In addition to the importance mathematically modelingthe nervous system has to physiological research is itsvalue to artificial intelligence and autonomous agents.

By employing spiking neural models as the process-ing elements for robotic agents, researchers are attempt-ing to explore theories that span the breadth and depthof robotics, AI and neuroscience. Understanding neu-rological processing often requires complex interactions

∗Corresponding Author. Corey@cmthibeault.com

with the real world (or a virtual one). This is the casein the fields of both social robotics [1, 2] and neuro-robotics [3–5]. These theories generally involve the inte-gration of several sensory modalities as well as complexcommands between the agent and the neural architec-tures controlling it. This integration can be a complextask requiring either expertise in both computer engi-neering and neuroscience or collaborations between ex-perts in each of these fields. The software, NCSTools,presented here was developed to ease the complexity ofinterfacing neural models with remote agents as well asabstract the neurological detail from the roboticists andthe engineering detail from the neuroscientists.

This paper is laid out with the remainder of thissection presenting a minimal background on spiking neu-ral models and neurorobotics as well as introducing NC-STools. This is followed by Section 2, which illustratesthe design choices made in this project as well as the ba-sic software engineering behind its implementation. Sec-tion 3 provides a complete example of how NCSTools canbe used to speed-up the task of interfacing with a spik-ing neural simulation. Finally, Section 4 concludes withsome current applications where NCSTools has been uti-lized as well as some future directions.

The work presented here is an extension of a presen-tation given at the Computational Neuroscience SocietyMeeting in San Antonio, Texas, August 2010 [6].

1.1 Spiking Neural Models

Simulating excitable cells involves integrating a set ofdifferential equations that describe the electrical activ-ity along the membrane of the cell. The unique aspectsof these cells is that once a threshold voltage has beenreached, an all or nothing avalanche of electrical currentoccurs. This results in a spike of electrical activity, oran action potential, and initiates the communication be-tween neurons. Its effect is felt by all neurons connectedto the one that spiked, or fired. Additionally, repeatedspike events can alter or grade the effect felt by thosedownstream neurons. This modification is referred to assynaptic level plasticity and is thought to play a majorrole in animal learning [7]. By combining the differential

equations describing membrane voltages with the synap-tic communication, computational neuroscientists hopeto reveal details of the nervous system unavailable withcurrent experimental techniques.

1.2 Neurorobotics

The field of neurorobotics focuses on the coupling of neu-ral systems with some form of physical actuation. Anexample of this is the concept of virtual neurorobotics(VNR). This is based around the interoperability of aneural model, a virtual robotic avatar and a human par-ticipant [8, 9]. Under all but the most basic scenariosthis interoperability is accomplished through an orga-nized network communication system. For this paperan emphasis is placed on robotic and automated agents;however, it should be noted that the tools described hereare by no means limited to that application.

1.3 The NeoCortical Simulator

The NeoCortical Simulator (NCS) was developed at TheUniversity of Nevada, Reno by the Brain ComputationLab under the direction of Dr. Phillip Goodman. Fromits inception a heavy emphasis has been placed on paral-lelization and performance. In addition, mechanism forgetting spiking information out and stimulus in was alsoextremely important. Despite the focus on performance,NCS provides a number of important biological models.For a review of what NCS has to offer refer to Wilsonet al. [10, 11] and to see how NCS compares to otherneural simulators see Brette et al. [12]. Its features canbe summarized as:

• High-performance MPI-based parallel architecture• Leaky Integrate-and-fire (LIF) neurons with con-

ductance based synapses and Hodgkin-Huxley chan-nels.

• Hebbian synaptic learning with Short-term plastic-ity, augmentation, and spike-timing dependent plas-ticity (STDP).

1.4 NCSTools

NCSTools is a bridge between neuroscientists and engi-neers using NCS for research. The strength of NCSToolslies in its configuration language. It provides a mecha-nism for defining interactions between the neural simu-lation and the agent. These interactions are describedusing plain text strings and developers simply need toagree on the strings to develop their components. Be-sides the benefit that neither end is required to knowintricacies of the other, this also provides a level of reusethat can reduce development time. The only require-ment is that the strings remain consistent. This greatly

reduces the development time of a neurorobotic applica-tion.

2 Design

NCSTools was developed in C++ with a focus on object-oriented design principles. The motivation for its con-struction was driven by the need for a replacement tothe previous software package, Brainstem [13]. Althoughsuccessful as a proof-of-concept, Brainstem lacked thenecessary extensibility and reliability required for rapiduse by researchers. Based on these inadequacies, sev-eral non-functional requirements were identified beforestarting the development of NCSTools. These were

1. Usability: The use of NCSTools must be relativelysimple. Although its intended users are scientistsand engineers, its operation has to make sense basedon the task.

2. Extensibility: The rapid pace of scientific researchnecessitates a system that can be readily extendedto incorporate new ideas and concepts.

3. Robustness: The codebase must be invariant tothe application and multitude of configurations.

4. Reliability: NCSTools is intended for researchersperforming experiments in both neuroscience androbotics. If it is unreliable, meaningful results canbe lost.

The component interfaces provide layered abstrac-tion supporting an extensible yet robust code base. Sim-ilarly, the configuration language supports overall usabil-ity. Throughout this section numerical subscripts thatcorrespond to the non-functional requirements are usedto indicate how each design element supports the re-quirements.

Figure 1 presents the system level layout of NC-STools. The major components are described below.

Figure 1: System layout. With the exception of the user inter-faces, all connections are made using the various network clientsdescribed below.

input :{

NCSReportCollection1 : {num reports = 500 ;# The t o t a l per iod o f counting , i n c l u d e s# repor t c o l l e c t i o n and recovery .per iod = 500 ;# The number o f r e p o r t s .NCSReports = 2 ;type = "STANDARD" ;NCSReport1 : {

connect ion = "to_PMC1" ;command = "point_left" ;} ;

NCSReport2 : {connect ion = "to_PMC2" ;command = "point_right" ;

} ;# Setup the p l o t s f o r t h i s group .p l o t = "YES" ;plotType = "BAR" ;plotname = "Motor Activity" ;tabIndex = 1 ;p lot Index = 1 ;

} ;} ;

Listing 1: Input configuration example. This is a fixedwindow input container with a window of 500 spike reports fromNCS. After that time the activity from the named connections“to PMC1” and “to PMC2” are compared. The commandcorresponding to the winning channel is sent to the connectionclients. For instance, if the “to PMC1” channel is more active thecommand “point left” is sent to the connected agents. The lastsection is used to configure if, where, and how this activity will bepresented on the user interface.

2.1 Configuration Language1

The flexibility of NCSTools lies in its configuration lan-guage. Most aspects are modifiable at runtime throughthe input configuration file. This includes control forthe definable communication “language”, the GUI, UserIO and Data processing. There are two examples of theconfiguration language provided in Listings 1 and 2. Thecomponent of NCSTools that these correspond to are de-scribed in Sections 2.2 and 2.3 respectively.

2.2 Inputs2,3

In the context of this paper, inputs are the signals com-ing from the NCS simulation specified as reports. NCSpublishes population level information about the simu-lation to each of the reports requested. This informa-tion can be spike-counts for individual neurons, sampledsynaptic weights and neuron voltages.

Input Containers

The base class for all inputs is the input container. Thisprovides the common functionality, such as initialization

and communication, as well as the interface for all de-rived classes.

Windowed Input Containers

Windowed input containers allow the user to specify awindow of time over which the spiking activity of anynumber of neuron populations can be compared in awinner-takes-all pattern. The window of time is deter-mined by how often the NCS simulation sends spikinginformation. There are currently two types of derivedwindow input containers, one uses a fixed amount of timeand the other uses a moving window.

The fixed time container compares the spiking ac-tivity over a static window. After the defined period ofNCS reports has elapsed the most active population (theone with the highest number of spikes) is selected. Thecommand specified in the configuration file for that re-port channel is then sent to all connected clients. Anexample configuration for a fixed time container is pre-sented in Listing 1.

The second input container uses a moving windowof time. This window is fixed width but progresses intime as the simulation progresses. This continues untilthe most active population’s activity is greater than thenext most active by a user specified threshold. At thattime the command associated with that report is sentout and the window is reset.

A key feature of both the inputs and the outputs,described in Section 2.3 below, is the ability to bind thesame command string to multiple containers. This pro-vides a mechanism for starting a coordinated series ofevents based on a single client command or simulationresult.

In addition to sending commands to the connectedagents these containers can also be used for plotting inthe GUI as described below.

2.3 Outputs2,3

The NCSTools outputs define the signals traveling to theNCS simulations. These are stimulus sent to populationsand can be triggered by text strings from the connectedclients or by the built-in touchpad interface describedbelow.

Static OutputThe stimulus sent to the neuron population is fixed anddefined by the configuration file.

Dynamic Output

The stimulus sent to the neuron population is sent bythe client along with the string command.

Timed Output

Timed output containers are used to send the same stim-ulus a set number of times. Through the configurationfile the user can specify how many times to send the

output :{

NCSStim1 : {type = "TIMED_OUTPUT" ;command = "saw_red" ;connect ion = "from_VC1" ;# The s t a t i c output to send .output = "0.2000" ;# How many t i c k s between stim inputs .f requency = 50 ;# The number o f t imes to repeat the input .num outputs = 1 0 ; ;

} ;NCSStim2 : {

type = "TIMED_OUTPUT" ;command = "saw_blue" ;connect ion = "from_childbot_VC1" ;output = "0.0000" ;f r equency = 50 ;num outputs = 1 0 ; ;

} ;} ;

Listing 2: Output configuration example. Two outputs areconfigured here; each tied to a different command from theconnected agents. These are both timed outputs where thestimulus, in this case a value of 0.2000, is sent every 50 simulationreports a total of 10 times. Where the stimulus is sent depends onthe string received from the connected clients.

stimulus and the interval between successive stimulus.An example of this is presented in Listing 2.

2.4 Network Communication3,4

Aside from the user interface, the connections illustratedin Figure 1 represent one or multiple network communi-cation mechanisms. NCSTools not only coordinates theconnection to and from the NCS neural simulation butalso provides a network server for handling client and re-mote agent connections. The individual components ofthe network communication provided by NCSTools aredescribed below.

2.4.1 NCSTools Server4

NCSTools uses a simple POSIX socket server for clientcommunication. The simplicity of the server helps ensureits reliability and the low-level components help guaran-tee that the performance of the server does not hinderthe overall application.

2.4.2 Client Communication2,3,4

Several client implementations are provided for C++,Python and MATLAB/Java. These provide objects thatappropriately abstract the interface from the implemen-tation to support the extensibility of both NCSToolsand client applications. Both blocking and non-blockingcommunication is supported.

IO Clients

The IO clients are used for most applications. Theseprovide the input and output mechanisms required tointeract with NCSTools.

Pacing Clients

As neural models increase in both size and complexitythey often exceed the real-time capabilities of the hard-ware. Pacing clients are provided to ensure that remoteclients do not overfill buffers or lose track of the sim-ulation. These connect to NCSTools to maintain syn-chronization and receive heartbeats that are output atuser specified intervals signaling the current time in thesimulation.

Passthrough Clients

Passthrough clients connect to the NCSTools server andreceive messages sent to or from other clients connectedto the server, including the NCS simulation. Theseclients can provide users with a complete landscape ofthe neurorobotic experiment. These can also be used tocreate context aware clients that can modify their be-havior based on the state of the system.

2.4.3 voServer

Communication with NCS is facilitated by the voServerpackage [14]. This is a minimal publish-subscribe serverthat provides both binary and ascii data streams. Aspart of this project a C++ client was developed similarto the NCSTools clients described above. This client isused by all of the IO modules of NCSTools.

2.5 Graphical User Interface

The Graphical User Interface (GUI) is an option givento users for visualizing aspects of the neural model inreal-time. The GUI was written as a C++ library usingQt [15]. As with all aspects of NCSTools, it is completelyconfigurable through the input file. The user can specifyeach tab and what information shows up on that tab.An example GUI is given in Figure 2.

Plot Types

The GUI is used to visualize the state of the NCS simu-lation. It provides three plot types:

Bar Plots

Bar plot are used to visualize the spiking activity of com-peting neuron populations. It uses the derived instancesof the windowed input containers, described in Section2.2.

Raster Plots

Raster plots present the precise spiking activity of theneurons within a population. The X axis is the time, in

Figure 2: Example graphical user interface. This is a single tabthat presents the three main types of plots. The tabs and plotsare dynamically created based on the configuration file providedat runtime.

units of simulation reports, and the Y axis correspondsto the neuron index relative to the population.

Line Plots

Line plots are used to plot the synaptic efficacy overtime, again this is in units of simulation time.

2.6 Touchpad Interface

The touchpad interface provides users with a way to bindkeyboard inputs, either single keys or new-line termi-nated strings, to stimulus and control signals. The inputis entered through the command line and is fully config-urable through the input language. There are three ma-jor options for the touchpad that are described below.The touchpad signals can be directed to a named NCSinput or to agents connected to the NCSTools server.Similar to the inputs and outputs described above, aparticular key binding can be used for any number ofdifferent commands. With this a single keyboard inputto control many different aspects of the neurorobotic in-teraction.

Instant: When touchpad bindings are defined as instanttheir associated actions will occur only once and as soonas the command is received.

Repeated: The repeated bindings are analogous to thetime outputs described in Section 2.3. These provide away to repeat an input stimulus over a configured periodof time.

Timed: The timed bindings provide a mechanism forsetting a period of silence before repeating the configuredstimulus. This basically creates a window of stimulusfollowed by a rest period. This window is then repeatedfor the configured number of times.

Coupling With Input Signals

The unique aspect of the Repeated and Timed bindingsis the ability to couple them to signals coming from theconnected agents. When a configured signal is receivedfrom a connected agent the user is prompted for a nu-merical value by the touchpad interface. This value isused to determine the graded input values sent to thesimulation. This is used in the example given in Section3 below.

2.7 Control Interface

In addition to sending stimulus and receiving reports,NCSTools provides access for controlling and modifyinga running neural simulation. Commands can be boundto strings from connected agents or to the built-in touch-pad functionality. Some of the features this releases in-clude the saving and loading of model states, modifica-tion of synapses, adding new stimulus paths and stop-ping the simulations.

3 Example Scenario

To illustrate NCSTool’s role in neurorobotic research,a motivating example is presented in Figure 3. In thiscase the remote agent is a virtual robotic avatar and theinteraction is with a camera and the touchpad interface.The steps are:

1. Camera captures image from user and dominantcolor is calculated.

2. The virtual environment sends the defined plain textstatement (“saw red”) to NCSTools through theserver interface. This uses a static output describedin Section 2.3.

3. Based on the configuration NCSTools will stimulatethe appropriate regions of the remote NCS Modelthrough the NCS network interface.

4. The activity of the two premotor regions in themodel are monitored and compared as the simula-tion progresses. A windowed input describe in Sec-tion 2.2 monitors the activity.

5. In this case a winner-takes-all calculation is com-puted and the appropriate plain text statement issent through the NCSTools server interface to therobotic avatar based on the most active brain re-gion.

VisualCortex

Premotor Cortex

NCS SimulationSTDP

Synapses

NCSTools

"saw red"

"point left"

12

3

4

5

0 10

6 Graded Reward

Figure 3: Example neurorobotic scenario.

6. The user is then given the opportunity to “reward”the robot if the correct color was identified. Thereward is achieved by coupling the output from theagent with the touchpad interface as described inSection 2.6.

Although this is a simple example there is still asignificant amount of coordination involved.

4 Discussion

NCSTools provides a dynamic interface to the neuralsimulation environment NCS. The abstraction betweenthe neural models and the robotic interface is unique tothis project and there have already been several projectsthat have successfully leveraged NCSTools.

Oxytocin Induced Trust

The work presented by Anumandla et al. [16] madeextensive use of NCSTools. In this project a humanparticipant interacted with a robotic avatar through aGPU based gabor processing application, NCSTools anda NCS simulation. The results of this work provided newtheories on the role Oxytocin may play in establishing

and stabilizing trust between mammals. These theorieswould not have been possible without a closed loop vir-tual neurorobotic system.

Emotional Speech Processing

Thibeault et al. [17] used NCSTools to coordinatethe processing between a speech extraction package anda neural simulation. The speech processing algorithmsuccessfully extracted the emotional aspects of a person’sspeech pattern to determine the reward stimulus to injectinto the model.

Virtual Reality

As part of an unpublished proof-of-concept, NC-STools was utilized for the large-scale visualization of arunning neural simulation. The neural model was con-structed within NCS along with a corresponding X3Dmodel. The visualization software created a virtual rep-resentation of the neuron populations. Through thenetwork interface the voltages of the cells within themodel were collected and as the simulation progressedthe model neurons would change color to represent thevoltage of the cell. In addition, 3D sound was used tosignal when a spike was fired. The package was testedsuccessfully on a 3D wall and a 6 sided Cave Automatic

Virtual Environment (CAVE).

Future Directions

There are several directions that have been identified forfuture development of NCSTools.

Real-time Structural Modifications

Providing a mechanism for users to modify aspectsof the model including the type of neuron and the con-nectivity, will greatly increase the rate at which differentmodels can be evaluated. In addition, this would pro-vide a mechanism for actively modifying neurogenesis(the addition of new neurons), and synaptogenesis (thedynamic addition and removal of synaptic connections).

Cluster-Aware Version

The complexity of large-scale neural models gener-ally requires distributed compute clusters. By creatinga cluster-aware version of NCSTools, multiple instancescan run in parallel while still coordinating communica-tion between instances.

Dynamic Server Configuration

The current version is only modifiable on initializa-tion. Allowing users to modify the configurable aspectsof NCSTools will make it a more appealing tool for neu-rorobotics.

AcknowledgementsThis work was supported in part by grants fromthe U.S. Defense Advanced Research Projects Agency(HR001109C001) and the U.S. Office of Naval Research(N000140110014).

References

[1] M. Scheutz, P. Schermerhorn, J. Kramer, andD. Anderson, “First steps toward natural human-like hri,” Autonomous Robots, vol. 22, pp. 411–423,2007. 10.1007/s10514-006-9018-3.

[2] K. Dautenhahn, “Socially intelligent robots: di-mensions of humanrobot interaction,” Philosophi-cal Transactions of the Royal Society B: BiologicalSciences, vol. 362, no. 1480, pp. 679–704, 2007.

[3] J. Krichmar, A. Seth, D. Nitz, J. Fleis-cher, and G. Edelman, “Spatial navigation andcausal analysis in a brain-based device modelingcortical-hippocampal interactions,” Neuroinformat-ics, vol. 3, pp. 197–221, 2005. 10.1385/NI:3:3:197.

[4] S. I. Wiener and A. Arleo, “Persistent activ-ity in limbic system neurons: neurophysiologicaland modeling perspectives,” Journal of Physiology-Paris, vol. 97, no. 4-6, pp. 547 – 555, 2003.

[5] N. Cuperlier, M. Quoy, and P. Gaussier, “Neuro-biologically inspired mobile robot navigation andplanning,” Frontiers in Neurorobotics, vol. 1, no. 0,2007.

[6] C. Thibeault, F. Harris, and P. Goodman, “Break-ing the virtual barrier: real-time interactions withspiking neural models,” BMC Neuroscience, vol. 11,no. Suppl 1, p. P73, 2010.

[7] D. Purves, G. J. Augustine, D. Fitzpatrick, W. C.Hall, A.-S. LaMantia, J. O. McNamara, and L. E.White, Neuroscience. Sinauer Associates, inc.,2008.

[8] P. H. Goodman, S. Buntha, Q. Zou, and S.-M. Das-calu, “Virtual neurorobotics (vnr) to accelerate de-velopment of plausible neuromorphic brain architec-tures,” Frontiers in Neurorobotics, 2007.

[9] P. H. Goodman, Q. Zou, and S.-M. Dascalu,“Framework and implications of virtual neuro-robotics,” Frontiers in Neuroscience, p. 5, 2008.

[10] E. C. Wilson, “Parallel implementation of a largescale biologically realistic neocortical neural net-work simulator,” Master’s thesis, University ofNevada, Reno, August 2001.

[11] E. C. Wilson, P. H. Goodman, and F. C. HarrisJr., “Implementation of a biologically realistic par-allel neocortical-neural network simulator,” pp. 1–11, 2005.

[12] R. Brette, M. Rudolph, T. Carnevale, M. Hines,D. Beeman, J. M. Bower, M. Diesmann, A. Morri-son, P. H. Goodman, F. C. Harris, Jr., M. Zirpe,T. Natschlager, D. Pecevski, B. Ermentrout,M. Djurfeldt, A. Lansner, O. Rochel, T. Vieville,E. Muller, A. P. Davison, S. El Boustani, andA. Destexhe, “Simulation of networks of spikingneurons: A review of tools and strategies,” Journalof Computational Neuroscience, pp. 349–398, Nov.2006.

[13] Q. Peng, “Brainstem: A neocortical simulator inter-face for robotic studies,” Master’s thesis, Universityof Nevada, Reno, December 2006.

[14] J. G. King, “Brain communication server: A dy-namic data transferal system for a parallel brainsimulator,” Master’s thesis, University of Nevada,Reno, May 2005.

[15] M. Dalheimer, Programming with Qt (2nd ed.).O’Reilly Media, 2002.

[16] S. Anumandla, L. J. Bray, C. M. Thibeault, R. V.Hoang, S. Dascalu, F. Harris, and P. Goodman,“Modeling oxytocin induced neurorobotic trust andintent recognition in human-robot interaction,” inInternational Joint Conference on Neural Networks,IJCNN, July 2011.

[17] C. M. Thibeault, O. Sessions, P. H. Goodman,and F. C. Harris Jr., “Real-time emotional speechprocessing for neurorobotics applications,” in Pro-ceedings ISCA’s 23rd International Conference onComputer Applications in Industry and Engineering(CAINE-2010), Las Vegas, NV., November 2010.