2D-environment navigation using a neural network
-
Upload
arturo-di-lecce -
Category
Education
-
view
192 -
download
0
description
Transcript of 2D-environment navigation using a neural network
![Page 1: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/1.jpg)
Cognitive Robotics Project
Di Lecce Arturo
POLITECNICO DI MILANO
2D-environment navigationusing a neural network
![Page 2: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/2.jpg)
Abstract
1. a robot with some kind of obstacle sensors
2. a controller for obstacle avoidance
The purpose of this project is simulating a robot that can navigate in a 2D environment avoiding obstacles
What we need:
Arturo Di Lecce POLITECNICO DI MILANO
![Page 3: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/3.jpg)
The robot
The robot is a bi-wheeled w/ circle shape rover equipped with:
Arturo Di Lecce POLITECNICO DI MILANO
Head Direction
Robot body (diam = 1 robot unit)
sensors range indicators
1) touch sensors2) range finder sensors
![Page 4: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/4.jpg)
The robot - touch sensors
Arturo Di Lecce POLITECNICO DI MILANO
two strips of touch sensors - one from -45° to 0° respect head direction (left)- one from 0° to 45° respect head direction(right)
LEFT TOUCH SENSORSdetects objects at 0.6 r.u
in [45°,0°]
RIGHT TOUCH SENSORSdetects objects at 0.6 r.u
in [0°,-45°]
![Page 5: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/5.jpg)
The robot - range finder sensors
Arturo Di Lecce POLITECNICO DI MILANO
two range finder sensors - one from -45° to 0° respect head direction (left)- one from 0° to 45° respect head direction(right)
LEFT RANGE FINDERSstarts to detect objects
at 11 r.u in [45°,0°]
RIGHT RANGE FINDERSstarts to detect objects
at 11 r.u in [0°,-45°]
![Page 6: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/6.jpg)
Neural network controller
Arturo Di Lecce POLITECNICO DI MILANO
“Learning Anticipation via Spiking Networks: Application to Navigation Control”
(Arena et al. - 2009)
● Speaks about a 2-layer neural network controller useful for robot navigation in 2D spaces with capability of obstacle avoidance and target approaching
• We focus on obstacle avoidance
![Page 7: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/7.jpg)
Neural network controller - the spiking network (1/3)
Arturo Di Lecce POLITECNICO DI MILANO
The neuron model used for this controller, is the one proposed by Eugene M. Izhikevich in his paper “Simple Model of Spiking Neurons” (2003):
![Page 8: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/8.jpg)
Neural network controller - the spiking network(2/3)
Arturo Di Lecce POLITECNICO DI MILANO
Depending on four parameters (a,b,c,d), the model reproduces spiking and bursting behavior of known types of cortical neurons
● a is the time scale of recovery variable u(t)● b is the sensitivity of recovery variable u(t)● c is the after spike reset value of membrane potential v(t)● d is the after spike reset value of recovery variable u(t)
![Page 9: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/9.jpg)
Neural network controller - the spiking network(3/3)
Arturo Di Lecce POLITECNICO DI MILANO
![Page 10: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/10.jpg)
Neural network controller – controller structure (1/3)
Arturo Di Lecce POLITECNICO DI MILANO
Composed by:● 2 unconditioned stimuli neurons UC● 2 consditioned stimuli neurons CS● a constant source of neural spikes Go-On● 2 motor neurons RMgo & LMgo● 2 motor neurons RMturn & LMturn
![Page 11: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/11.jpg)
Neural network controller – controller structure (2/3)
Arturo Di Lecce POLITECNICO DI MILANO
UnconditionedStimuli
neurons
Go-OnMotor
Neurons
TurnMotor
Neurons
ConditionedStimuli
neurons
![Page 12: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/12.jpg)
Neural network controller – controller structure (3/3)
Arturo Di Lecce POLITECNICO DI MILANO
2 kind of inputs to 2°nd layer neurons:
● excitatory synapses (like USL → RMTURN) marked with an arrow● Inhibitory synapses (like USL → RMGO) marked with a dot
![Page 13: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/13.jpg)
Neural network controller – controller integration
Arturo Di Lecce POLITECNICO DI MILANO
Left Touch Sensor
Right Touch Sensor
Left RangeFinder
Right RangeFinder
Right Motor
LeftMotor
![Page 14: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/14.jpg)
Neural network controller – controller inputs
Arturo Di Lecce POLITECNICO DI MILANO
If the distance between the robot and an nearest obstacle is
<= 0.6 robot units
● Unconditioned stimuli & touch sensors
With this function, the range finders sensors approximately starts to fire at
a distance of 11 robot units (d0 is obstacle distance)
● Conditioned stimuli & range finders
![Page 15: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/15.jpg)
Neural network controller – neuron input
Arturo Di Lecce POLITECNICO DI MILANO
The synaptic input to a generic neuron J is given by:
Where:● wij represents the weight of the synapse from neuron i to neuron j
● ti is the instant in which a generic neuron i connected to neuron j, emits a spike
● The function ε(t) is
![Page 16: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/16.jpg)
Neural network controller – controller outputs
Arturo Di Lecce POLITECNICO DI MILANO
A single motor input is the sum of:● # of spikes emitted by GoOn neuron● # of spikes emitted by Turn neuron
![Page 17: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/17.jpg)
Neural network controller – synaptic weights learning
Arturo Di Lecce POLITECNICO DI MILANO
Synaptic weights for CS-->TurnMotors are continuously learned according to the STDP rule:
Where Δt is the difference between the spiking time of the presynaptic neuron and that of the postsynaptic one and the rest are parameters of the learning algorithm.
For avoiding that the weights increase steadily, weight decay has been introduced:
![Page 18: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/18.jpg)
Neural network controller – controller behavior (1/2)
Arturo Di Lecce POLITECNICO DI MILANO
● The motor driver signal depends on the number of spikes in the output neurons assigned to the motor
● GoOn motor neurons generate the spike train needed to let the robot advance in the forward direction
● The spikes of the Turn motor neurons are then summed to the ones of the GoOn neurons
● In presence of collisions, GoOn neurons are inhibited and the forward movement is suppressed
● When the left and right motor neurons emit an equal number of spikes, the robot moves forward with a speed proportional to the number of spikes. In absence of conditioned stimuli, the amplitude of the forward movement is about 0.3 r.u. for each step
FORWARD MOVEMENT
![Page 19: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/19.jpg)
Neural network controller – controller behavior (2/2)
Arturo Di Lecce POLITECNICO DI MILANO
● When there is a difference in the number of spikes emitted by left and right motor neurons, the robot rotates. The angle of rotation (in counterclockwise direction) is:
θ = 0.14*Δnswhere
Δns = nr - nl
● We count the spikes emitted both by the left and the right neuron and the robot advance with a speed proportional to the minimum number of them
ROTATION
![Page 20: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/20.jpg)
Simulation - Intro
Arturo Di Lecce POLITECNICO DI MILANO
Main simulation function:
Parameters:● posn: start position of the robot [x,y,θ];● map_name: relative path of the map image (.png)● steps: #of simulation steps● w: initial weights (if 0, weights are initialized into function)● debug: flag for debugging and showing some graphs● drawPath: flag for drawing the robot movements at the end of the simulation
Returns:● posn: final position of the robot● w: weights at the end of the simulation
![Page 21: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/21.jpg)
Simulation – neural network representation
Arturo Di Lecce POLITECNICO DI MILANO
Membranepotentials
Recoveryvariables
Inputs
![Page 22: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/22.jpg)
Simulation – some settings
Arturo Di Lecce POLITECNICO DI MILANO
![Page 23: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/23.jpg)
Simulation – soft sensor
Arturo Di Lecce POLITECNICO DI MILANO
offset=π/2delta=π/2
![Page 24: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/24.jpg)
Simulation – neuron input function
Arturo Di Lecce POLITECNICO DI MILANO
![Page 25: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/25.jpg)
Simulation – setting inputs
+ if excitatory input- if inhibitory input
Touch sensors input
Range finders input
Arturo Di Lecce POLITECNICO DI MILANO
![Page 26: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/26.jpg)
Simulation – weigths update
Arturo Di Lecce POLITECNICO DI MILANO
![Page 27: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/27.jpg)
Simulation – controller (1/2)
Arturo Di Lecce POLITECNICO DI MILANO
Retrieve LM & RM go and turn neurons number of spikes
Go forward
Rotate
![Page 28: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/28.jpg)
Simulation – controller (2/2)
Arturo Di Lecce POLITECNICO DI MILANO
Calc new position
Update distances
![Page 29: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/29.jpg)
Simulation 1 – Results (1/3)
Arturo Di Lecce POLITECNICO DI MILANO
First simulation: 200 timesteps
weights have not yet been learned!
![Page 30: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/30.jpg)
Simulation 1 – Results (2/3)
Arturo Di Lecce POLITECNICO DI MILANO
If touch sensors detect an obstacle, forward movement is inhibited and the
robot turns
![Page 31: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/31.jpg)
Simulation 1 – Results (3/3)
Arturo Di Lecce POLITECNICO DI MILANO
![Page 32: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/32.jpg)
Simulation 2 – Results (1/3)
Arturo Di Lecce POLITECNICO DI MILANO
After 1600 timesteps
From 1600to 2300
timesteps
![Page 33: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/33.jpg)
Simulation 2 – Results (2/3)
Arturo Di Lecce POLITECNICO DI MILANO
Weights are learned and if range
sensors detects an obstacle, the robot
turns
![Page 34: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/34.jpg)
Simulation 2 – Results (3/3)
Arturo Di Lecce POLITECNICO DI MILANO
Weights learning curves
![Page 35: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/35.jpg)
Simulation – Some problems (1/2)
Arturo Di Lecce POLITECNICO DI MILANO
Weights reach their maximum/minimum valuesafter some k-simulation steps
In Arena & al. simulation, this doesn't happen!
![Page 36: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/36.jpg)
Simulation – What can be wrong?
Arturo Di Lecce POLITECNICO DI MILANO
Simulation result depends on a lot of factors:
● Neuron model parameters (a,b,c,d) – influence spike rate, sensitivity and default membrane potential
● STMP parameters (A+,A-,Tau+,Tau-) – influence weights learning rate
● Simulation step: lower values, increases the precision of spike time variables so the weights are learned better but the simulation time grows
● Robot's speed● Soft touch/range sensor algorithm● Different simulation environment (in Arena et al. they used C++)
![Page 37: 2D-environment navigation using a neural network](https://reader033.fdocuments.us/reader033/viewer/2022052621/557cbc75d8b42a1b0c8b4f20/html5/thumbnails/37.jpg)
Simulation – Goin' further ...
Arturo Di Lecce POLITECNICO DI MILANO
● It's possible to integrate target approaching controls adding an inter-neurons layer to the controller and a visual input sensor (a camera)