Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna...
-
Upload
earl-sulton -
Category
Documents
-
view
213 -
download
1
Transcript of Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna...
![Page 1: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/1.jpg)
Research in Intelligent Mobile Robotics (and related topics)Part 1: Navigation and Vision
Anna Helena Reali [email protected] www.pcs.usp.br/~anna
Laboratório de Técnicas Inteligentes Escola Politécnica da Universidade de São Paulo
Carlos Henrique Costa [email protected] www.comp.ita.br/~carlos
Divisão de Ciência da Computação Instituto Tecnológico de Aeronáutica
![Page 2: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/2.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 2
Preface
This is a two-part talk about the research on Intelligent Mobile Robotics and related topics at: LTI – USP (Laboratório de Técnicas Inteligentes –
Universidade de São Paulo, Brazil) NCROMA-ITA (Laboratório de Navegação e Controle
de Robôs Móveis Autônomos – Instituto Tecnológico de Aeronáutica, Brazil).
These research groups are involved in the MultiBot cooperation project CAPES/GRICES with ISLab-IST.
![Page 3: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/3.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 3
LTI - EPUSP
Prof. Anna Reali 5 PhD Students
Alexandre Simões*, Reinaldo Bianchi, Valdinei Silva*, Valguima Odakura, Waldemar Bonventi.
3 Master Students Alexandre Cunha*, Antônio Selvatici, Luiz Carlos Maia
3 Undergraduate Students Rafael Pacheco, Márcio Seixas, Júlio Kawai
2 Final Course Projects
![Page 4: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/4.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 4
NCROMA – ITA
Prof. Carlos Ribeiro 1 PhD Student
Letícia Friske
5 Master Students Luís Almeida, Ricardo Maia, Juliano Pereira,
Esther Colombini*, Celeny Alves*
2 Undergraduate Students Lucas Gabrielli, Fábio Miranda
![Page 5: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/5.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 5
Intelligent Mobile Robotics
Part 1:Navigation• Map building• Localization
Perception• Computer Vision
Part II:Learning
Contents
![Page 6: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/6.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 6
Map Building
Our goal: Test map building algorithms in real robots fast enough? precise enough? ok for learning applications (e.g. path
learning)?
![Page 7: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/7.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 7
Map BuildingSildomar Takahashi, Carlos RibeiroRoberto Barra, Ricardo Domenecci, Anna Reali
“Efficient Learning of Variable-Resolution Cognitive Maps for Autonomous Indoor Navigation”.
Arleo, Millan e Floreano, IEEE-SMC, 1999. Advantages:
Complete algorithmic description Simple structure
Limitations: Assumes structure (orthogonal obstacles / walls) Reliance on dead-reckoning (but can be adapted to
more sophisticated localization)
![Page 8: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/8.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 8
The Basic Algorithm
1. Explores environment;
2. Once an obstacle is detected:
1. Determines obstacle frontiers either via: An a priori sensor model
A pre-trained neural net
2. Includes obstacle in the global map;
3. Defines new partition to explore. If there is none, END. Else finds route to new partition.
4. Executes route and explores new partition. Once an obstacle is detected, Step 2. Else, Step 1.
![Page 9: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/9.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 9
Detection of obstacle frontiers
Robô
Células Ocupadas
RetaCalculada
• Either a priori model or neural net model• Integration over time• Straight-line adjustment and correction (according to a priori actuator model)
0 y
x
My
Mx
![Page 10: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/10.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 10
Very “Scientific” Set-up
Walls Obstacles
Robot
3 x 3,5 m
![Page 11: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/11.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 11
Global Maps: Magellan, Neural net model
Map 1
Map 2
Map 3
![Page 12: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/12.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 12
Global Map: Pioneer, a priori model + straight-line model-based correction
![Page 13: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/13.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 13
Conclusions Tested algorithm (possibly with some
modifications) is a good compromise efficiency/precision for realistic applications: fast yet fairly accurate.
Next steps: Studies on simultaneous localization and
mapping (SLAM algorithms). Valguima Odakura (Anna Reali): SLAM based
on visual landmarks. Fabio Miranda (Carlos Ribeiro): Bayesian
landmark learning. Techniques for map building acceleration.
![Page 14: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/14.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 14
Markov LocalizationLuís Almeida, Carlos RibeiroJúlio Kawai, Anna Reali
Position estimation based on Bayesian update: Belief update based on sensor info Belief update based on action info
Sensor/actuator models and initial belief distribution: arbitrary.
Simple to implement.Computationally costly (Monte Carlo
implementation – particle filters – is a possible fix).
![Page 15: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/15.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 15
Markov Localization
X
p (x )
b )
X
p (x )
d )
X
p (x )
h )
X a)1
X c)1
X e)1 X2
X
p (x )
f)
X g )2
Probabilistic position grid
Action Model
Sensor Model Markov State
Estimator
pt
st
at
pt+1
odometerssonars
![Page 16: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/16.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 16
Monte Carlo Localization + GA OptimizationLuís Almeida, Carlos Ribeiro
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)
Standard Markov update (over set of particles)
MC MC MCGA GA
GA on population of particles (fitness as combination of belief / particle cluster distribution)
GA on population of particles (fitness as combination of belief / particle cluster distribution)
• Basic idea: use GA to create a better set of particles for next MC update.
• Initial results: ok (in need of statistical validation).
![Page 17: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/17.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 17
Next Steps
Validation of GA approach. Better sensor and actuator models. Implementation in a real robot. Literature on Monte Carlo methods
(applications on signal detection and tracking): many variations to be tried...
![Page 18: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/18.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 18
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
![Page 19: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/19.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 19
Color Classification - I
Using threshold values: In the color representation space
Neural Network – MLP + backpropagation alg.:Alexandre Simões, Anna Reali
![Page 20: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/20.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 20
Derived Application:Alexandre Simões, Anna Reali
C1 C2 C3 C4 C5
Orange Classifier
Branco
. . .
Verde Claro
Verde Escuro
Amarelo
Laranja Claro
Laranja Escuro
R
G
B
Danificado
Orange Classifier - CEAGESP, SP
![Page 21: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/21.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 21
For number of clusters = 2 to Cmax, do: Apply FCM-GK (non-supervised fuzzy classifier)
to RGB image; Calculate the ratio c/s for each cluster set:
c = Cluster dimension/number of members s = Separation among clusters
Choose the cluster set, based on c/s. Show color classification result for the best
cluster set.
Non-supervised iterative fuzzy color classificationWaldemar Bonventi, Anna Reali
![Page 22: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/22.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 22
An example: soccer
The best cluster set 6 clusters
![Page 23: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/23.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 23
Another example: Rio de Janeiro
The best cluster set 3 clusters
![Page 24: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/24.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 24
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
![Page 25: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/25.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 25
Background Model - I
Background subtraction Thresholding the error between an estimate of
the image without moving objects – M(C) – and the current image:
Model can not adapt to environment changes!
M(C)Current Image
![Page 26: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/26.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 26
Background Model – IIMárcio Seixas, Anna Reali
Time-Adaptive, Per-Pixel Mixture-of-Gaussians: Time series of observations at a given pixel (its color) is
modeled by a mixture-of-gaussians. Based on the persistence and the variance of each of the
gaussians of the mixture, it is determined which gaussians may correspond to background colors.
Hypothesis: gaussian distributions with low variance and high persistence correspond to background model.
Per-pixel models are updated as new observations are obtained (according to a learning rate).
It is capable of dealing with long-term scene changes (e.g. lighting changes)!
![Page 27: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/27.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 27
Derived Application:platform occupancy
Fixedmodel M(C):
Original:
Original – M(C):
Adaptive Model:
Terminal Rodoviário de Santo Amaro TRENDS & Prefeitura de São PauloMárcio Seixas, Anna Reali
![Page 28: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/28.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 28
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
![Page 29: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/29.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 29
Optical Flow - idea
![Page 30: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/30.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 30
Robot (with camera) navigating in a stationary scenario. Calculation of the optical-flow divergent to estimate the time-
to-crash value in order to avoid collisions with obstacles. We are now investigating a robust method to directly
calculate the per-pixel time-to-crash value:
Vision-based robotic behavior:Antonio Selvatici, Anna Reali
Gray levels: near bright; far darkBlack: unknown distance
Original sequence Pixel time-to-crash Filtered values
![Page 31: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/31.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 31
Derived Application: monitoring of underground rail tracksLuiz Maia, Anna Reali
ALSTOM & Metrô de São Paulo
![Page 32: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/32.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 32
Computational Vision
Image Segmentation Using Color Classification Using Background Model Using Optical Flow Based on Binocular Stereo Vision
![Page 33: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/33.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 33
Distance-Map Calculation1. Calibration [Zhang, ICCV 99]
2. Matching – blob coloring+centroid+correlation
3. Triangulation Segmentation: based on color + distance-map.
Binocular Stereo VisionRafael Pacheco, Anna Reali
![Page 34: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/34.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 34
Derived Application:Outdoors Measurement
TRENDS & Prefeiturade São PauloRafael Pacheco, A. Reali
![Page 35: Research in Intelligent Mobile Robotics (and related topics) Part 1: Navigation and Vision Anna Helena Reali Costa anna.reali@poli.usp.br anna.](https://reader035.fdocuments.us/reader035/viewer/2022070306/5518cfc9550346b31f8b5c1f/html5/thumbnails/35.jpg)
MultiBot - Meeting #1, Lisboa 2003 - part I 35
Conclusions In CV, we are now investigating:
Automatic learning of fuzzy color classifiers – Waldemar Bonventi, LTI;
A framework for high-level feedback to adaptive, per-pixel, mixture-of-gaussian background models – Márcio Seixas, LTI;
Mathematical formulation for direct and robustly calculate the per-pixel, time-to-crash values, considering a moving observer in a stationary scenario – Antonio Selvatici, LTI;
Distributed, real-time approach to calculate the optical flow, considering a stationary observer in a dynamic scenario – Luiz Maia, LTI.