A study of two complementary encoding strategies

based on learning by demonstration for autonomous

navigation task

Antoine de Rengerve∗ Florent D’halluin∗∗ Pierre Andry∗

Philippe Gaussier∗ Aude Billard∗∗

∗ETIS, CNRS ENSEA University Cergy-Pontoise F-95000 Cergy-Pontoise∗∗LASA, EPFL, CH-1015 Lausanne, SWITZERLAND

Abstract

Learning by demonstration is a natural andinteractive way of learning which can be usedby non-experts to teach behaviors to robots.In this paper we study two learning by demon-stration strategies which give different an-swers about how to encode information andwhen to learn. The first strategy is based onartificial Neural Networks and focuses on re-active on-line learning. The second one usesGaussian Mixture Models built on statisticalfeatures extracted off-line from several train-ing datasets. A simple navigation experimentis used to compare the developmental possibil-ities of each strategy. Finally, they appear tobe complementary and we will highlight thateach one can be related to a specific memorystructure in brain.

1. Introduction

Human development is clearly influenced by interac-tions performed during the whole life. One of thenatural way to teach someone how to do somethingis simply to demonstrate what is expected from him.Programming by demonstration [Billard et al., 2008]is a key approach for development in autonomousrobots. It does not require any technical knowledgefrom the teacher as it tries to provide the robot withlearning abilities similar to those present in humanbeings. This is particularly interesting when deal-ing with robotic systems that must perform a widerange of tasks. Approaches requesting to programevery possible behavior happen to be a dead-end.

In this work, demonstrations enable the robot tobuild behaviors based on the learning of sensorimo-tor associations. The system can then infers whatto do when presented with some given sensory infor-mation (proprioception, vision). However, there aredifferent ways to encode such sensorimotor associa-tions, and different ways to learn them, which willinfluence the developmental capacities of the robot.

In this paper, we will focus on two strategies thatgive distinct answers to the question of encoding andlearning. The Neurocyber team of ETIS lab devel-oped a system based on artificial Neural Networksthat allows a user to teach a mobile robot how tonavigate robustly using visuo-motor association [La-garde et al., 2010]. Meanwhile, the LASA lab usesa statistical approach based on Gaussian MixtureModels to learn the sensorimotor coupling. This canbe employed for teaching gestures to robotic arms orhumanoid robots [Calinon et al., 2009].

A simple U-shaped navigation task has been cho-sen in order to compare these two approaches. Therobotic platform used in this work, a Robulab fromRobosoft, can select a direction to go forward. Whenthe robot goes away from the right path, its direc-tion can be modified by using a joystick. In order tocompute its position, the robot can use a monocularpan moving camera. With pan rotation, the cam-era provides the robot with visual panorama for self-localization. An odometer can be used for recordingthe trajectory in the Cartesian space.

The Neural Network (NN) model and the GaussianMixture Models (GMM) that can perform naviga-tion task will be respectively developped in Section2 and Section 3. The presented experiment illus-trates similarities and differences between these twoapproaches. These two systems enable a robot tolearn actions in the context of learning by demon-stration in interaction with a human teacher. Bothsystems are based on state-action associations butthey do not learn and encode information in the sameway. Section 4 discusses the consequences of thesedifferent encodings for the capacities of the systemwhile considering memory-cost, adaptation throughlong time learning, interaction and quality of the tra-jectory. In Section 5, we discuss the complementaryaspects of these two approaches and how they couldbe related to different kinds of memories as humancognition is concerned. They show similarities withthe Hippocampus and the Neocortex as describedin [McClelland et al., 1995].

place cell

Visual processing

target azimut

GMM

target azimutMagnetic Compass

Discrete State action coupling

place cellContinuous State action coupling

Figure 1: Overview of the system: Once the visual pro-

cessing is done on data, it activates continuously the place

cells neurons.

Figure 2: Model of place-movement associations learning.

“what” and “where” information is extracted from vision

and proprioception. They are merged and compressed in

place codes allowing place recognition. These place codes

are associated to proprioceptive informations.

2. Using Neural Networks for naviga-tion

Neurocyber team has developed a controller for mo-bile robots which is able to associate visual informa-tion (a panorama of the environment) with self ori-entation (using a compass representing the directionof the actual movement). This controller is designedas a sensorimotor loop based on a neurobiologicalmodel testing some of the spatial properties of thehippocampus [Giovannangeli et al., 2006].

2.1 Place cells definitions

To be able to localize itself and navigate, the robotuses the recognition of place cells based on visualcues (Figure 1 and [Giovannangeli et al., 2006]for more details about visual features extraction).A place is defined by a constellation of visualfeatures (landmark-azimuth couples) extracted froma panorama (Figure 3) compressed into a place code.The visual system extracts local views centeredon points of interest (landmark recognition) thatprovides information of “what”. A magnetic com-pass acting as proprioceptive information (spatiallocalization in the visual field) provides informationof “where”. The place code results of the mergingof “what” and “where” information.

The merging of the information is performed ina product space (i.e. a matrix of product neuronsmk(t) called merging neurons) defining a place codeM(t). More details about the definition of theplace code and the merging neurons can be found

Figure 3: Example of visual features extraction on half

visual panorama. The system computes a gradient on

each view. 4 visual features are extracted from each gra-

dient.

in [Lagarde et al., 2010]. The place-cell activitiesare built as the result from the computation of thedistance between the learned place code and thecurrent place code. The activity PCp(t) of the pth

place cell is:

PCp(t) =1Wp

(nM∑k=1

ωPCkp (t)mk(t)

)(1)

where ωPCkp (t) expresses the fact that the landmark-azimuth couple k (i.e. the kth merging neuron whichactivity is mk(t)) has been used to encode the placecell p. The number of couples used by the pth placecell is given by Wp =

∑nM

k=1 ωPCkp , with nM the num-

ber of recruited neurons in the landmark and az-imuth matrix. A place cell learned in the locationA responds maximally in A and creates a large de-creasing field around A. Such a system is able to learnseveral regions of the environment. It can performvisual localization.

2.2 Place-movement association to definetrajectories

Originally, the robot follows a random direction.When the robot takes a wrong direction, the humanteacher can use a leash to drag the robot towardthe desired path. When doing so, the user modifiesthe dynamics of the robot and the motor commandof the wheels orientates the robot in the desireddirection. The resulting change in proprioception(orientation change) triggers the learning of theassociation between the movement being done andthe visual panorama (the current location). Theorientations and the speeds are discretized so thateach neuron Si corresponds to a given orientationand a given speed. Their activities are calculatedwith (2).

si(t) =nP C∑p=1

ωSpi(t) · PCp(t)

Si(t) = V (t) · Sdi (t) + (1− V (t)) ·(

si(t)smax(t)

)(2)

When the teacher modifies the dynamics of therobot, the change is detected and the vigilance signal

a.

dynamical attractor

trajectory

associated place-movement b.

Figure 4: Learning of a path by correction of the learned

dynamics. Each arrow represents a correction applied

by the teacher. Hence, the robot learns the association

between the place and its orientation. a.) A straight

line trajectory can be defined by only two state-action

associations. This trajectory is an attractor as from every

point in the space, the robot will reach the line. b.) An

example of trajectory that can be learned. After 3 rounds

of learning, the robot is fully autonomous, the professor

does not need to correct it anymore.

V becomes equal to 1. smax = maxi=1..nS

(si) is used

for an output normalization with nS the numberof actions. The output Si can either be the actionpredicted by the place cells or the desired actionSdi (t) (orientation and speed) that is determinedfrom the action performed by the robot with orwithout the intervention of the teacher. The weightωSpi of the connection between the pth place cell andthe ith action is adapted according to a learningrate ε(t) and the learning rule (3) which is inspiredof Widrow&Hoff gradient descent rule:

dωSpidt

= (Sdi (t)− si(t)) · PCp(t) · V (t) · ε(t) (3)

Building sensorimotor attractors by associatingmovements to different regions of the environment( [Giovannangeli and Gaussier, 2010], Figure 4) en-ables the robot to reproduce the learned trajec-tory. A more sophisticated version of this associa-tive learning exists which is not described in thispaper [Giovannangeli and Gaussier, 2010]. In thecase of the short learning of the navigation exper-iment described in Section 2.3, that version showslittle difference with the presented version.

2.3 Application of the model to a U-shapedtrajectory

The robot follows a direction which is corrected bythe human teacher whenever it is too far from the de-sired trajectory. Modifications of the dynamics of therobot imply the learning of new place cells (Figure5). The Neural Network model enables the reproduc-tion of this simple trajectory after only three runs,

−2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0x [m]

−0.5

0.0

0.5

1.0

1.5

2.0

2.5

y [m

]

12

3

4

5

6

7

8

9

10

11

12

13

12

3

Figure 6: Experimental data corresponding to a simple

U-shaped path. Top: trajectories in the Cartesian space

(using odometry, reset at the beginning of each run).

Grey lines are the trajectories during which learning oc-

curred. The successive starting position are numbered.

The reproduction (no correction) trajectory is in black.

Dark blue arrows are the learned place-orientations.

using only the corrective interaction from the humanteacher (Figure 6). During each run, the followingdata are recorded: activations of place cells (outputsfrom the Neural Network), the current position andorientation of the robot (using both odometry andmagnetic compass). These data are used to trainthe GMM-based system for the navigational task.

3. Gaussian Mixture Model ap-proaches for robot navigation

In this section we propose two uses of Gaussian Mix-ture Models in order to learn trajectories in naviga-tion task. The first implementation is based on anideal situation with relevant data directly available.Secondly, we propose an implementation which usessubjective visual cues in a more autonomous and re-alistic approach.

3.1 A direct transposition of the manipula-tion model using Cartesian coordinates

A Gaussian Mixture Model defines a probability den-sity function on the state space of the robot.

p(ξ) =K∑k=1

πk1√

(2π)D|Σk|e−

12 ((ξ−µk)>Σ−1

k (ξ−µk))

(4)D is the dimension of the state space and k is thenumber of states. πk are prior probabilities, µk aremean matrices, Σk are covariance matrices and ξare points of the state space. The GMM is char-acterized by the three parameters πk, µk,Σk. Given

0 10 20 30 40 50time steps

0.0

0.2

0.4

0.6

0.8

1.0

plac

e ce

lls a

ctiv

atio

n

1 2 3 4

0 10 20 30 40time steps

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Figure 5: Experimental data corresponding to a simple U-shaped path. Left: Robulab mobile platform used for the

experiments. Middle: place cell activation during trajectory while learning phase. Peaks correspond to the creation of

new place cell, when user sets a new target azimuth (sharp turns in the Cartesian space) Right: place cell activation

during the reproduction phase.

a training set of n datapoints (ξi), an Expectation-Maximization algorithm is used to find the parame-ters πk, µk,Σk that maximizes the likelihood (6) ofthis training set. Expectation: We first estimatefor each point of the training set the probability thatthis point ξi is generated by each Gaussian (or statek ) p(k|ξi).

p(k|ξi) =p(ξi|k)p(k)∑Kj=1 p(ξi|j)p(j)

(5)

where p(ξi|k) = N (ξi, µk,Σk) and p(k) = πk. Theoverall likelihood of observing the given training setwith the given model parameters is:

L(x) =n∏i=1

(K∑k=

p(ξi|k)πk

)(6)

This is used as measure of convergence and perfor-mance of this algorithm. Maximization: Meansand covariances of each Gaussian are recomputed byweighting each data point by the probability p(k|ξi).These two steps are iterated until convergence.

The Gaussian Mixture Regression (GMR) enablesto probabilistically complement partial information.Let ξ = [ξOξI] be a point in the state space with ξI

that is known. The GMR enables to estimate ξO bytaking the mean of the expectation distribution (7)(see [Cohn et al., 1996] for more details).

p(ξO|ξI) ∼K∑k=1

p(k|ξI) p(ξO|ξI, k), (7)

We now consider that the state space is ξ =[θ v x y] with θ the absolute azimuth, v the linearvelocity and x, y the Cartesian position of the robot.The data recorded during the three training runs ofthe NN system (see Section 2.3) are used to generatethe training sets (ξi) for the Gaussian Mixture Model

(GMM). The training is off-line. A simulation of arobot using the orientation commands retrieved fromthe GMR has been realized. The resulting trajectorywith a 4 states GMM is shown in Figure 7. Thissimulation tells us that it makes sense to use sucha continuous state action model to learn trajectoriesfor a differential drive robot.

The drawback of this approach is that the abso-lute Cartesian position of the robot must be known,which in this case was obtained by the carefully re-calibration of an odometer. This approach can beused only if the robot has access to the absoluteCartesian position. Odometry is not reliable for thatpurpose. Without a regular recalibration, the possi-ble drift in the computation of the position can leadto an important inaccuracy. A robust localizationsystem is necessary, such as a statistical localizationsystem (Kalman Filter, Particle filter [Thrun et al.,2001]), or an external visual tracking system with ac-curate calibration. These methods are still costly tosettle, since they require either extra computation,or extra hardware.

3.2 Navigation with GMM training usingplace-cell activity

In a more developmental and autonomous approach,the robot should rely on subjective visual cues to lo-calize itself. This would allow more robustness andadaptation as the robot would not need a predeter-mined map of the environment to localize itself. Theraw landmark-azimuth pairs introduced in the neu-ral approach in Section 2.1 could be used. Though,as the dimension of the inputs increases, the com-putational time for the GMM model increases likeO(n3). Moreover, the structure of the visual in-formation with noise and occulting implies that thenumber of pertinent information may not be stabi-

2.5 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5x [m]

0.5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

y [m

]

12

3

Figure 7: Generated trajectory (black), and model

using information on the absolute Cartesian position

(p(θ, v|x, y)). Dashed lines are demonstration data,

GMM states are represented with green ellipses and the

light gray field lines represent target azimuths at given

position.

lized which makes the training more difficult.Place cells offer an efficient pre-processing of the

visual inputs for the training. In the Neural Networkmodel, they have proved to be robust to noise and oc-cultation (keeping the same ordering), and they rep-resent a smaller amount of information. The mod-ule that calculates the place-cell activities is addedto the Gaussian Mixture Model system so that thestate-space is now defined as ξ = [θ(t) PC(t)] withPC(t) = (PC1(t), · · · , PCN (t)). The number ofplace cells is chosen arbitrarily. At every time step wecan retrieve a continuous value for the target azimuthusing (7) which becomes equation (8). It defines acontinuous and probabilistic mapping between sen-sory inputs PC(t) and the motor command θ(t).

θ(t)→ p(θ|PC(t)) (8)

The three training runs realized using the NeuralNetwork system (Section 2.3) provide the trainingdata. At the end of these runs, twelve place cellshad been learned. The GM Model is trained using8 Gaussians and these 12 place cells. A Gaussiannoise of variance 0.1 is added to place cell activationsduring the training to simulate noisy visual inputs.Place-cell activities from the test set with the Neu-ral Network model are given to the GMM to predictazimuth target. In Figure 8, a comparison betweenthis azimuth and the real one from the test run withthe Neural Network system shows that using place-cell activations is acceptable for retrieving a targetazimuth from the GMM. Because place cells are gen-erated during turns, there are no place cells at thebeginning of trajectories. That can explain why the

0 20 40 60 80 100 120time step

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

5.5

azi

mut

[rad]

measured azimutpredicted azimut (using 12 pc and 8 gauss)

Figure 8: Predicted azimuths using place cell activation

based on Equation 8 and using the data from 12 place

cells. The surface in light blue represent the incertitude

on the target angle given by the probability density (±σ).

model performs badly at the beginning.

The previous experiment allows us to compare theretrieved orientation command on a test trajectory.We now simulate place cells by Gaussian activations.The parameters of the Gaussian activations of theplace cells can be estimated from the recorded dataprovided by the learning and test runs for the Neu-ral Network system. It is then possible to associateplace-cell activities to every position in the environ-ment. These activities are given to a simulated robotto reproduce the learned trajectory. Three extrapo-lated place cells are used. The different parts of thetrajectory correspond to different dominant states ofthe GMM. It appears that the states defines a newtopology that is built on top of the place cells. Theresult of the simulation is given in Figure 9. Usingsuch an approximation shows that the GMM basedsystem is well able to reproduced the desired trajec-tory by using radial basis activation for place cells.

4. Encoding and learning strategies

Both approaches succeed in solving the task. How-ever they present differences in encoding that influ-ences the abilities of the system. The Neural Net-work system is based on vision. It gathers informa-tion (azimuth, landmarks) into place cells which areassociated to desired orientations when the robot tra-jectory is corrected. The Gaussian Mixture Modelencodes statistically pertinent information. UsingCartesian coordinates as inputs is efficient for navi-gation, but place cells can also be used.

Figure 9: Simulated trajectory with a GMM system us-

ing place-cell activities approximated by Gaussians. The

light gray ellipes display simulated place cells. Black thin

lines are training trajectories. The reproduction trajec-

tory is represented by successive points. For each point,

the most probable Gaussian among the four in the Gaus-

sian Mixture Model gives its design to the point. There

are four different designs, one for each Gaussian state.

4.1 Optimized encoding and long timelearning

With several runs and corrections whenever the NNsystem is not showing the required behavior, a dy-namical attractor can be constructed (Figure 4).Each time the trajectory of the robot is corrected,another place cell is learned. But there is no guar-antee that the new place cell will only correct themistake. As place cells have a wide recognition ac-tivity, new place cells can interfere with older ones.The attractor stability and generalization rely on thenumber and the positions of place cells recruited tolearn the actions [Giovannangeli and Gaussier, 2010].Several place cells can be needed to stabilize the dy-namics over the desired path, especially under strongconstraints (if small variance on the resulting trajec-tory is needed). With many place cells, there can bea redundancy in the encoded information. A specificimplementation for adaptive orientation cells can en-hance the model [Giovannangeli and Gaussier, 2010].Before recruiting new place cells, it is possible totry to adapt the orientation associated to an alreadyknown cell. However, the optimized repositioning orpruning of place cells is currently an unsolved prob-lem. As learning continues, with changes of the envi-ronment, more and more place cells may be recruitedleading to an oversampling of the space.

Generalizing from several samples enables theGMM system to optimize the encoding of the tra-jectory by computing Gaussian states that mini-mize redundancy and maximize available informa-tion. There is no risk of bad interference as every

state participates during learning. With the testedimplementation, the number of Gaussians that willbe used by the system must be given in advance.Other specific implementations can solve this prob-lem and enable incremental learning [Calinon et al.,2009]. As learning continues, the first datasets maybecome obsolete. If the environment changes toomuch, the system requires a complete re-learningfrom new correct data.

4.2 Interaction

The presented Neural Network implements on-linelearning by correction of the behavior. The robotcan reproduce immediately a learned walk, even if itmay not exactly be the one that is shown. As theteacher can directly see what has been learned by therobot, bad orientation can be instantaneously cor-rected. This new association immediately modifiesthe dynamics of the robot which can be correctedagain, etc. This incremental approach enables thesystem to focus on ill-learned part of the trajectory.

The learning of the GMM system can be qualifiedof slow as the robot must first be driven completelypassively before it can do anything. The whole tra-jectory must be shown several times so that the robotget enough training data. The learning is off-lineand requests several samples of data so that statisticsabout the task can be built. Each sample requiresthe teacher to realize a trajectory from start to endso they are tedious for the human teacher. The en-coding depends on the complete training. What waslearned by the robot can not be known before theend of the training. If the robot must be corrected,another set of demonstrations by the human teacheris necessary to retrain the system.

4.3 Quality of the learned trajectory

The trajectory generated using GMM-based systemis a smoother approximation of the U-shaped desiredtrajectory than the one generated with the NeuralNetwork system. In the Neural Network system, apossible solution to avoid interferences is to use dis-cretized orientations and a competition between theplace cells. When using a strict competition, thetrajectories often appear as straight concatenatedlines, and do not seem very natural. A soft com-petition over the recognized place cells suppress thisproblem (a few winner place cells activate a mixtureof associated directions allowing smoother trajecto-ries). In the GMM system, since both inputs andoutputs are continuous, when the system is betweentwo known positions, it automatically combine therequired behaviors yielding nicer trajectories. Be-cause the actions are demonstrated by a human, thedesired trajectory is hidden by variations. The sta-tistical analysis of the datasets enables the GMM to

retrieve the optimal trajectory and it can even de-termine the constraints and degrees of freedom fora given walk. According to the variance of the ex-pectation distribution, the trajectory can be more orless constrained.

4.4 Explicit and implicit encoding, fast andslow learning

The difference in encoding can be summarized byexplicit versus implicit encoding. The NN systemdirectly encodes the state-action associations thatare perceived during learning. The GMM systemdistributes the implicit encoding between differentstates. No state is specifically related to an event.This difference of encoding is bound to a difference inlearning. Explicit associations can be learned rapidlyand independently like corrections in the NN system.On the contrary, implicit associations require severaldata. The learning is slowed down by the need of sev-eral passive demonstration to create several trainingdatasets. The system can not reproduce any actionuntil the end of this training.

5. Discussion

The context of this study is learning in a situationof interaction. More specifically, we focused on twolearning by demonstration approaches that are basedon state-action associations. We do not tackle rein-forcement learning as it does not really correspondto demonstration by a human teacher. The robotcan not explore its environment to build reinforce-ment evaluation. It must use the information pro-vided by the interaction. Linking interaction andreinforcement is an ongoing work (see [Hirel et al.,unpublished]).

In [Zukowgoldring and Arbib, 2007], the authorsdefend the hypothesis that the main feature of inter-action between an infant and its caregiver is not thedirect transfer of knowledge but the reduction of theresearch space in order to help the infant to learnnew tasks. The fast corrective learning in the NeuralNetwork system corresponds quite well to this defi-nition. As the lively space is reduced, the robot isled to reproduce what the human teacher wants it todo with more or less accuracy. But, this is limited.Rather than increasing more and more the numberof corrective rules that define the action, a statisti-cal analysis of the structure of the possible actionscould optimize the encoding. In the case of a robot,the GMM system could complement the Neural Net-work system to enhance the learning abilities.

This reflexion driven by a developmental point ofview is also based on an anatomical analysis. [Eichen-baum et al., 1994] stresses the complementarity oftwo distinct structures in the brain that are the Neo-cortex and the Hippocampus. These structures have

specificities which are very similar to the specifici-ties of the two encoding and learning strategies thatwe study in this paper. The Hippocampus is an ex-plicit associative memory that can acquire rapidlyinformation. As there is little inference from the en-coded items, the interferences are limited. It is alsorelated to novelty detection like changes of orienta-tion in the case of our robot. This structure cor-responds well to the place-cell associations learnedin the Neural Network architecture. The Neocor-tex is an implicit memory as it does not directly en-code specific events. It can discover gradually thestatistical structures of experiences, by accumulat-ing learning and data. This structure corresponds tothe GMM process. Motor and PreMotor Cortex canbe the location where the invariant features of move-ments and trajectories are retained. Some substruc-tures and other structures of the brain participatesin specific manners in the cognitive process. The En-torhinal Cortex interfaces the hippocampus and theneocortex. In [Arleo and Gerstner, 2000], EntorhinalCortex is the location of the place cells. Place-cellactivities can be used by both structures and en-able exchanges between Hippocampus and Neocor-tex. This corresponds to the system we simulatedin Section 3.2. The cerebellum provides interpo-lation and prediction abilities to the brain system.With interpolation and conditioning, it can improvethe quality of movements. There exist a transfer ofknowledge between the hippocampal episodic shortterm memory and the neocortical long term mem-ory [McClelland et al., 1995]. This happens mainlyduring sleeping time. The cerebellum can play animportant role in this process. Its predictive capac-ities can be used for internal rehearsal of episodicmemories so that Neocortex can learn implicit rep-resentation. This long term representation can thenbe used by the Striatum to generate routine move-ments. During the transfer between short term andlong term memory, a change in representation, ie.of encoding, can occur. This process is called mem-ory consolidation. It has been observed in animalsand in human [McClelland et al., 1995]. In [Kulicand Nakamura, 2009], memory consolidation is usedwith incremental learning so that the system canlearn on-line with additive stability coming from con-solidation memory which occurred both on-line dur-ing wake time and off-line during equivalent sleepingtime. This consolidation enabled a better catego-rization of the action. This memory consolidationhas also proved to make human-robot collaborationeasier in [Ogata et al., 2004].

In this paper, the NN system and the GMM sys-tem have been studied separately. As a future work,we suggest that they are gathered in a whole archi-tecture that benefits from the complementarity of thetwo strategies. Once the robot has acquired a new

rough behavior, it can reproduce the trajectory overand over providing the necessary datasets for GMMtraining. Learning can be done in two times: a firstrough learning of the task which is then refined usingmore data to determine the invariant features of theactions that solves the task. One model can take overthe second one according to the situation. When fac-ing already known situations, the GMM based sys-tem can produce an optimized adapted behavior andwhen facing new situations, the Neural Network en-ables interactive corrections to incrementally gener-ate an adapted behavior.

The discussion provided here has been based ona navigation experiment, but the conclusion shouldbe extended to action selection problem in general.We believe that the same explicit/implicit memorystructures can be involved not only in navigation,but also in manipulation tasks or even higher com-plex tasks mixing different kinds of behaviors. Fu-ture works will study this hypothesis. As it can beseen in Section 4, the conclusions drawn may not berestricted to the algorithms studied in this paper.The conclusions should be extended to algorithmsthat could be classified either as fast learning, ex-plicit encoding or as slow learning, implicit encoding.

In conclusion, the two strategies studied in thispaper must be considered as two complementary en-coding and learning strategies. One can not replacethe other one, it is not a matter of trade-off be-tween the two strategies. According to how evolu-tion built our brain, both strategies must be presentin order to provide the cognitive system with mostefficiency as considering reactivity and interactivityduring learning with good inferences to optimally re-tain any demonstrated knowledge.

6. Acknowledgement

This work was supported by the French RegionIle de France, the Institut Universitaire de France(IUF), the FEELIX GROWING european projectand the INTERACT french project referenced ANR-09-CORD-014.

References

Arleo, A. and Gerstner, W. (2000). Spatial cogni-tion and neuro-mimetic navigation: a model ofhippocampal place cell activity. Biological Cy-bernetics, 83(3):287–299–299.

Billard, A., Calinon, S., Dillmann, R., and Schaal,S. (2008). Survey: Robot Programming byDemonstration. In Handbook of Robotics, vol-ume chapter 59. MIT Press.

Calinon, S., Caldwell, D., Billard, A., andD’halluin, F. (2009). A probabilistic approach

to learn and reproduce dynamics of human mo-tion by imitation. In Proceedings of 2009 IEEEInternational Conference on Humanoid Robots.

Cohn, D. A., Ghahramani, Z., and Jordan, M. I.(1996). Active learning with statistical models.CoRR, cs.AI/9603104.

Eichenbaum, H., Otto, T., and Cohen, N. J. (1994).Two functional components of the hippocampalmemory system. Behavioral and Brain Sciences,17(03):449–472.

Giovannangeli, C. and Gaussier, P. (2010). Interac-tive teaching for vision-based mobile robots: Asensory-motor approach. IEEE Transactions onMan, Systems and Cybernetics, Part A: Systemsand humans, 40(1):13–28.

Giovannangeli, C., Gaussier, P., and Desilles, G.(2006). Robust mapless outdoor vision-basednavigation. In IEEE/RSJ International Confer-ence on Intelligent Robots and systems, Beijing,China. IEEE.

Kulic, D. and Nakamura, Y. (2009). Incremen-tal learning and memory consolidation of wholebody human motion primitives. Adaptive Be-havior - Animals, Animats, Software Agents,Robots, Adaptive Systems, 17(6):484–507.

Lagarde, M., Andry, P., Gaussier, P., Boucenna, S.,and Hafemeister, L. (2010). Proprioception andImitation: On the Road to Agent Individuation,volume 264 of Studies in Computational Intelli-gence, pages 43–63. Springer Berlin / Heildel-berg.

McClelland, J. L., Mcnaughton, B. L., and O’Reilly,R. C. (1995). Why there are complementarylearning systems in the hippocampus and neo-cortex: insights from the successes and failuresof connectionist models of learning and memory.Psychol Rev, 102(3):419–457.

Ogata, T., Sugano, S., and Tani, J. (2004). Open-end human robot interaction from the dynami-cal systems perspective: Mutual adaptation andincremental learning. In Innovations in AppliedArtificial Intelligence, pages 435–444. SpringerBerlin / Heidelberg.

Thrun, S., Fox, D., Burgard, W., and Dellaert, F.(2001). Robust monte carlo localization for mo-bile robots. Artificial Intelligence, 128(1-2):99–141.

Zukowgoldring, P. and Arbib, M. (2007). Af-fordances, effectivities, and assisted imitation:Caregivers and the directing of attention. Neu-rocomputing, 70(13-15):2181–2193.