TFC Sensor Fusion– Pedro Marcelino LEIC 27 de Fevereiro de 2003 Sensor Fusion Applied to Soccer...

27 de Fevereiro de 2003 TFC “Sensor Fusion”– Pedro MarcelinoTFC “Sensor Fusion”– Pedro Marcelino

LEIC

Sensor FusionApplied to Soccer Robots

Sensor FusionApplied to Soccer Robots

Prepared by: Pedro MarcelinoOriented by: Prof. Pedro Lima


LEIC

• Motivation• Sensors Caracteristics• Sensors as Members of a Team• Sensor Models• Observation Integration• Implemented Algoritms• Experimental Results• Conclusions

“Sensor Fusion”TopicsTopics


LEIC




LEIC

• Increased interest in the developing of multi-sensor robots

• Limitations in the reconstruction of environments

• Observation errors, bad calibrations or partial and incomplete information of the world

• Cooperation to resolve ambiguities

• Robust and consistent description of the world

• Team with a common goal and shared knowledge, so it can take the right decisions.

“Sensor Fusion”MotivationMotivation


LEIC




LEIC

• Sensor Complexity• Observation Error• Observation Disparity• Multiples Points of View

“Sensor Fusion”Sensors CaracteristicsSensors Caracteristics


LEIC




LEIC

• Multi-Sensorial System = Team of Sensors

• Each sensor is considerer an individual

• Each sensor make local decisions

• Each sensor implements its actions

• The Team coordinate the activity of its members

• Information exchange to resolve conflits and validation of observations

• Makes the Team Decision Problema a simple Estimation Problem

“Sensor Fusion”Sensor as a Team MemberSensor as a Team Member


LEIC




LEIC

• Observation Model It is a static description of the sensor performance,

realting the observation with the state of teh environment

Front Camera ModelUp Camera Model

“Sensor Fusion”Sensor ModelsSensor Models

CL

C


LEIC

“Sensor Fusion”Front Camera Observation ModelFront Camera Observation Model

Modelo da Câmara da Frente

0

0.20.4

0.60.8

11.2

1.4

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5

Distância (m)

Var

iân

cia

(m)

Modelo da Câmara da Frente XX Modelo da Câmara da Frente YY


LEIC

“Sensor Fusion”Up Camera Observation ModelUp Camera Observation Model

Modelo da Câmara da Cima

0

0.05

0.1

0.15

0.2

0.25

0 0.5 1 1.5 2 2.5 3 3.5

Distância (m)

Var

iân

cia

(m)

Modelo da Câmara da Cima XX Modelo da Câmara da Cima YY


LEIC

• State ModelRelates the observation of a sensor with a given

location and its internal statePerspective change to a common frame so that the

observation can be compared



LEIC

• Each feature is represented as with a gauss distribuition

• Mean

• Variance

• Angle with central axis

• Distance to feature

• New variance results from the perspective transformation to a global frame

“Sensor Fusion”State ModelState Model


LEIC

• Dependency ModelDescribe sthe relation between the observations and

the actions of each sensorTeam Utility FunctionTeam Decision FucntionGroups Rational AximosEach member makes a decision that maximizes its

Team Utility Function



LEIC




LEIC

“Sensor Fusion”Observation IntegrationObservation Integration

• Each feature is modeled by a gauss distribution, using Bayes Law

• If the Mahalanobis distance is less than 1, then there is agreement and the team member will cooperate, to estimate the feature position, otherwise, there is desagreement and the team member observation will not be used


LEIC• Two bayes observers showing agreement

“Sensor Fusion”Observation IntegrationObservation Integration


LEIC• Two bayes observers showing desagreemnet

“Sensor Fusion” Observation IntegrationObservation Integration


LEIC




LEIC

• Ball detection in front Camera

“Sensor Fusion”Implemented Algoritms – Ball DetectionImplemented Algoritms – Ball Detection


LEIC

• Ball detection un Up Camera

“Sensor Fusion” Implemented Algoritms – Ball DetectionImplemented Algoritms – Ball Detection


LEIC

“Sensor Fusion”Camera ModelsCamera Models

Comparação das Posições Observadas pelas Câmaras e a Posição Real da Bola

ao longo de uma Linha Recta

-0.15

-0.05

0.05

0.15

0 0.5 1 1.5 2 2.5 3 3.5 4

Eixo XX (m)

Eix

o Y

Y (

m)

Posição Câmara Cima Posição Câmara Frente Posição Real

Comparação dos Erros de Leitura segundo Eixo XX ao longo de uma Linha Recta

0

0.2

0.4

0.6

0.8

0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.75 3 3.25

Distância ao Robot (m)

Err

o (

m)

Erro Câmara Cima XX Erro Câmara Frente XX

Comparação dos Erros de Leitura segundo Eixo YY ao longo de uma Linha Recta

00.020.040.060.080.1

0.12

0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.75 3 3.25


Err

o (

m)

Erro Câmara Cima YY Erro Câmara Frente YY


LEIC

“Sensor Fusion”Sensor Models DiagramSensor Models Diagram

Observation Model

State Model

Observation of Sensor 2

Dependency Model

Team Utility Function

Sensor Model

Change of perspective to Global Frame

Decision and Integration of Observation

Structure that keeps all decisions made by the

team members

New Fusion Validation

Variance Increase with Time

Observation Model

State Model

Observation of Sensor 1


LEIC




LEIC

“Sensor Fusion”Experimental ResultsExperimental ResultsComparação da Trajectória da Bola com a Fusão e dados Observados ao longo de uma Linha Recta

-0.15

-0.05

0.05

0.15

0 0.5 1 1.5 2 2.5 3 3.5 4

Eixo XX (m)E

ixo

YY

(m

)

Posição Câmara Cima Posição Câmara Frente

Posição Real Posição Fusão

Comparação dos Erros segundo Eixo XX ao longo de uma Linha Recta

0

0.2

0.4

0.6

0.8

0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.75 3 3.25


Err

o (

m)

Erro Câmara Cima XX Erro Câmara Frente XX Erro Fusão XX

Comparação dos Erros segundo Eixo YY ao longo de uma Linha Recta

00.02

0.040.06

0.080.1

0.12

0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.75 3 3.25


Err

o (

m)

Erro Câmara Cima YY Erro Câmara Frente YY Erro Fusão YY


LEIC




LEIC

“Sensor Fusion”ConclusionsConclusions

• Real time fusion of the world information

• Good estimative of features localization

• Makes system more robust, eliminating sporadic errors

• Coerent World decription

• Use of Bayes Teorem to solve the decision problem

• It is a really good method to be used in modern robotics, which should be used whenever possible to determine the position and orientation of the environment features that surrond the robot


LEIC

“Sensor Fusion”Future WorkFuture Work

To be developed during the Master

• Sensor Fusion of several robots

• Other players detection

• Team players detection

• Sensor Fusion to determine robot position


LEIC

“Sensor Fusion”Sensor Fusion DiagramSensor Fusion Diagram

Sensors Up Camera Front Camera Sonars Odometry

Observation and State Model

Up Camera Front Camera Sonars Odometry

BlackBoard local.up.* local.front.* local.sonars.* local.odometry.*

Dependency Model

Local Sensor Fusion Algoritm

BlackBoard global.worldmodel.* World Model

Global Sensor Fusion Algoritm

Dependency Model

Local Sensor Fusion Algoritm of Other Robots


LEIC

Team Members

• Docentes do IST:• Pedro Lima (coordenação) - DEEC• Luis Custódio (coordenação) - DEEC• Carlos Pinto Ferreira (professor associado) - DEM

• Alunos de Doutoramento (EEC):• Miguel Garção

• Alunos Finalistas (TFC):• Bruno Damas - LEEC• Pedro Pinheiro - LEIC• Hugo Costelha - LEEC• Gonçalo Neto - LEEC• Cláudio Gil – LEIC• Miguel Arroz – LEIC• Bruno – LEIC

TFC Sensor Fusion– Pedro Marcelino LEIC 27 de Fevereiro de 2003 Sensor Fusion Applied to Soccer...

Documents

Transcript of TFC Sensor Fusion– Pedro Marcelino LEIC 27 de Fevereiro de 2003 Sensor Fusion Applied to Soccer...