Robot Vision with CNNs:a Practical Example
P. VitulloP. Campolucci
G. ApicellaL. Pompeo
D. BellachiomaS. Graziani
M. BalsiDep. of Electronic Engineering
“La Sapienza” Univ. of Rome, Italy
X. Vilasís–CardonaS. LuengoJ. SolsonaR. Funosas
A. MaraschiniA. Aznar
V. GiovenaleP. Giangrossi
Barcelona, 19/2/03
Framework of this work
• completely autonomous robot• simple (cheap) hardware• vision-based guidance
– short term: line following– longer term: navigation in a real environment
Architecture
• Cellular Neural Networks to handle all the image processing
• Fuzzy-rule-based navigation
Cellular Neural Networks
• Fully parallel analog vision chips• Capable of real-time nonlinear image
processing and feature detection
• Algorithmically programmable to implement complex operations
• On-board image acquisition (focal-plane processing)
Cellular Neural Networks
• Recurrent Neural (?) Network• Locally connected VLSI-friendly• Space-invariant synapses (cloning
templates)– small number of parameters: explicit design
• Continuous variables – analog computing (discrete-time model for digital)
TopologyLocally connected VLSISpace-invariant synapses
Discrete–time model
• Binary state variable• Analog or binary input depending
on implementation
IuB
nxAsignnx
ijNklklljki
ijNklklljkiij
;
;1
Application• Input ports: analog arrays u, x(0)• Output port: binary array x()• “Analog instruction”: {A,B,I} (cloning
template)• Feature detection (nonlinear image
filtering)
IuB
nxAsignnx
ijNklklljki
ijNklklljkiij
;
;1
CNN “Universal” Machine
• Local memory• Global control (broadcasting cloning
templates and memory transfer commands)
• “Analogic” computing: stored-program analog/logic algorithms
Task: line following
• The robot is to follow a maze of straight lines crossing at approximately right angles
• Functions required by vision module:
Acquiring image, cleaning, thinning linesMeasuring orientation/displacement of lines
Image processing algorithm
• Image acquisition
• Binarization
• Line thinning
Image processing algorithm (ctd.)
• Directional line filtering
• Projection
Fuzzy control
Simulation
y (m) z vs. x (m)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
el cochecito(Barcelona)
control (386)
CNN emul. (DSP)
Visibilia (Rome)
PAL B/WCAMERA
FPGA-based CNN emulatorCeloxica RC-100 board
Xilinx Spartan II 200Kgates
microcontroller
Jackrabbit BL1810
PIC 16F84
SERVOMOTOR
(steering)
LCD
PS/2 mouse port
Rabbit2000microcontroller
Parallel port E
Parallel port ASerial port D
STEPPERMOTOR
(advancing)
STEPPER MOTOR
CONTROLLER
Celoxica RC-100
VGA
Jackrabbit BL1810
drivingstart
vert
hor
follow vert
horY
Y
N
N
horY
N
normal driving
crossing
timer:=0
timer>10s N
Y
store left avail.
turn left if avail.else right
diag (L/R)
Y
follow diagY
N
Continuation of the work
more realistic tasks:• obstacle avoidance• navigation in a real-life environment
Obstacle avoidance• using other sensors together with
vision, e.g. ultrasound• monocular range evaluation• local path-finding strategies
Hybrid (topological/metric) navigation
door recognition
Robot Vision with CNNs:a Practical Example
M. BalsiDep. of Electronic Engineering
“La Sapienza” Univ. of Rome, Italy
Barcelona, 19/2/03
Top Related