Vision Control of Mobile Robots

7/29/2019 Vision Control of Mobile Robots

1/40

P R E S E N T E D B Y P U L K I T S H A H( 2 0 1 1 E E A 2 2 3 8 )

VISION CONTROL OF A

MOBILE ROBOT

Guide:Dr. Shubhendu Bhasin


2/40

PROBLEM STATEMENT

Mobilerobotwith

camera

Designing a control law todrive the robot from positionA to position B using onlyimages as inputs

A B


3/40

PROBLEM STATEMENT

Feature points

Mobilerobotwithcamera Epipoles


4/40

COMMON APPROACH TO SOLVING

PROBLEM

Pose based visual servoing

Velocity controller


5/40

A BRIEF LOOK INTO THE PBVS

APPROACH: HOMOGRAPHY

*

cos sin 0

sin cos 00 0 1

( ) [ ( ) ( ) 0]

* ( *) **

( *)*

i i

T

x y

T

i

i i

T

m Rm q

R

q t q t q t

d n mm Hm

qH R n

d

Sovle forR ,

, n*


6/40

AMBIGUITY IN HOMOGRAPHY

Solution to H is not unique, we get 8solutions.

We can eliminate four of them by putting constraint that the robot must

always be in front of the plane.

From the remaining four we can eliminate two by putting a constraint on

it must be between 0 and 2 Now we are left with two solutions, both of which satisfy all the above

constraints.

For resolving this ambiguity we must have some prior knowledge of the 3-

D scene in which our robot is operating

For example, if all the feature points are taken on the z-plane then the

normal vectorn* would be [0 0 1]T


7/40

THE IBVS APPROACH

Image based visual servoing


8/40

SOME BASIC CONCEPTS IN VISION


9/40

PERSPECTIVE CAMERA MODEL

x

y

Xp f

Z

Yp fZ

*

0 0

0 0

0 0 1

p K P

f

K f

( , )x yp p


10/40

EPIPOLES


11/40

HOW DO WE FIND EPIPOLES?


12/40

EPIPOLAR CONSTRAINT


13/40

EPIPOLES FROM ESSENTIAL MATRIX

Epipoles are just the right and left nullspaces of the Essential matrix

00

T

d

a

e EEe


14/40

COMPUTING THE ESSENTIAL MATRIX, E

Computing E from Point Matches Assume that you have m

correspondences Each correspondence satisfies:

E is a 3x3 matrix (9 entries) We get a HOMOGENEOUS linearsystem with 9 unknowns

0, 1,...,Tri lip Ep i m

B


15/40

11 12 13

21 22 23

31 32 33

11 21 31 12

22 32 13 23 33

( , ,1)

( ', ',1)

0; 1,...,

' ' 1

1

' ' '

' ' ' 0

T

li i i

T

ri i i

Tri li

i

i i i

i i i i i i i

i i i i i

p u v

p u v

p Ep i m

e e e u

u v e e e v

e e e

u u e u v e u e v u e

v v e v e u e v e e

COMPUTING E

where m>7


16/40

COMPUTING E

11

1 1 1 1 1 1 1 1 1 1 1 1

12

13

21

22

23

31

32

33

' ' ' ' ' 1

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .0

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .' ' ' ' ' 1m m m m m m m m m m m m

eu u u u u v u v v v u v

e

e

ee

e

e

eu u u u u v u v v v u v

e

A minimum least squares problem


17/40

WHY EPIPOLES?

When the actual and desired configurations are

aligned, both epipoles coincide with the origin ofthe corresponding image frame.

Epipoles =0Robots have same orientation!


18/40

INTUITIVE OUTLINE OF CONTROL

STRATEGY

Step 1:Use acontrollaw todrive therobotsuch thatepipoles

go tozero

Step 2:Use afeature-basedcontroller toeliminate thetranslationerror.

A


19/40

STRATEGY

Mobilerobotwithcamera

Actualposition Desired

position

Step 2:

Translating todesired position

Intermediateposition

Step 1: Aligning theactual and desiredviews


20/40

cos

sin

x v

y v

s = [x y ]T

PROBLEM FORMULATION: DEFINING A

NONHOLONOMIC KINEMATIC MODEL


21/40

FIRST STEP: ZEROING THE EPIPOLES

Deriving epipole kinematics:

( , ) ( , , )au due e f x y

( , ) ( , )au due e f v

Note that the v component of epipole coordinate (u,v)will be zero at all times as the robot is moving in aplane


22/40

GEOMETRICAL SETUP

sin cos

cos sin

du

au

x

e fy

x ye f

x y


23/40

EPIPOLE KINEMATICS

2 2 2 2

2 2

2 2

( )

( )

au auau du auau

au duau duau

au

e e fsign e e e fe v

d f f

e e fsign e ee vd f e f

We need to design v and such that eauand edu go to zero, the obvious choice is:

1 au

du

evD

e

au

du

e v

De


24/40

SOLUTION APPROACH: ROADBLOCK

au

du

e vD

e

2 2 2 2

2 2

2 2

( )

( )

0

au auau du au

au duau du

au

e e fsign e e e f

d f fD

e e fsign e e

d f e f

Distance between the actual and thedesired robot position) is unknown in apurely image-based control setting.

A


25/40

SOLUTION APPROACH

It is possible to us an approximate inverse of D bysetting

2 2

2 2

1

2 2 2 2

0 ( )

( )

au

au du

au du

au du

df e f sign e e

e e fDf f

e f e f

11

2

vv

D v

Where an estimate ofd,

has been used

A


26/40

SOLUTION APPROACH

2 2

2 21 11

2 2

1 1

0

au

au du

du

e fde v vv d e f

D DDe v v

d

d

The resulting epipole velocities are:


27/40

DESIGNING CONTROL INPUTS

2 2/

1 22 2

1 auau au dudu

e fde k e k e

d e f

/

2

du du

de k e

d

11

/

22 .

au

du

k ev

k ev

Let:

Closed loopepipole

dynamicsare:


28/40

CHOOSING UPDATE EQUATION FOR

ESTIMATE

2 2/

1 22 2

1 auau au dudu

e fde k e k e

d e f

Closed loopepipole

dynamicsare:

/

2

du du

de k e

d

Needs to be positive in order for edu to converge to zero

A

B


29/40


ESTIMATE

xx yyd

d

/2

2 2 2

( )

du

au du

ed k f d e e f

B

0

0 0 0

0

0

. 1 1

.

t

t

d d dt d dd

d dd d dt

We can choose any

value of 0>d

0

B


30/40


ESTIMATE

B

Epipoles converge to zero in finite time !

/2

2 2 2

( )

du

au du

ed k f d

e e f

B


31/40

SECOND STEP: MATCHING THE

FEATURES

the desired and the actual epipoles are zero andthe intermediate robot configuration qi is alignedwith the desired configuration

We need a control law which will make thetranslational error to zero.

Let be the norm difference betweenthe actual and the desired projection of the featurepoint and

B

2 2|| || || ' ||D p p

0

tv K D


32/40

PROOF: SECOND STEP

'

TX Y

p f fZ Z

TX Y

p f fZ d Z d

2 2 2 2 2

2 2

(2 )|| || || ' || ( )

( )

d Z dD p p f X Y d

Z Z d

2 21 12 2

V d d

2. t tV d v k dD k d

Globally

Exponentially

Stable!


33/40

CONTRIBUTION


34/40

SIMULATION AND RESULTS

Step 1

Step 2


35/40

SIMULATION: ERROR IN X AND Y

COORDINATES

xe(t)ye(t)


36/40

SIMULATION: ERROR IN ORIENTATION

AND DEPTH ESTIMATE

Error in(t)


37/40

SIMULATION: LINEAR AND ANGULAR

VELOCITIES

Step 2Step 1

Linear velocity, v Angular velocity,

B


38/40

ADVANTAGES OF THIS APPROACH

It is an IBVS hence no prior info needed of the 3-Dscene

As we use epipoles singularities stemming from

inversion on Jacobian are avoided As we do not need to decompose a homography

matrix, ambiguity in solutions is avoided

IBVS more robust to camera calibration than PBVS

where we do not rely on the calibration matrix toextract epipoles from the images.

B


39/40

FUTURE WORK

Extend the algorithm to discrete time system

to account for the sampling rate of camera

acquiring images(in real world scenario)

Extracting the feature points from a givenimage

To implement the strategy on a mobile robot

Combining IBVS and PBVS methods toeliminate the issue of singular configurations

in IBVS (2 D visual servoing)


40/40

QUESTIONS

Vision Control of Mobile Robots

Documents

Transcript of Vision Control of Mobile Robots