MSU CSE 803 Fall 2008 Stockman1 CV: 3D sensing and calibration Coordinate system changes;...

MSU CSE 803 Fall 2008 Stockman

1

CV: 3D sensing and calibration

Coordinate system changes; perspective transformation; Stereo and structured light


2

roadmap using multiple cameras using structured light projector 3D transformations general perspective

transformation justification of 3x4 camera model


3

Four Coordinate frames

W: world,

C,D: cameras,

M: object model

Need to relate all to each other.


4

Can we recognize?

Is there some object M

That can be placed in some location

That will create the two images that are observed?

Discover/compute what object and what pose


5

Need to relate frames to compute

relate camera to world using rotations and translations

project world point into real image using projection

scale image point in real image plane to get pixel array coordinates


6

Stereo configuration

2 corresponding image points enable the intersection of 2 rays in W


7

Stereo computation


8

Math for stereo computations need to calibrate both cameras to

W so that rays in x,y,z reference same space

need to have corresponding points find point of closest approach of

the two rays (rays are too far apart point

correspondence error or crude calibration)


9

Replace camera with projector

Can calibrate a projector to W easily. Correspondence now means identifying marks.


10

Advantages/disadvantages of structured light


11

Grid projected on objects

All grid intersects are integral


12

Computing surface normals

Surface normals have been computed and then added to the image (augmented reality)


13

Relating coordinate frames need to relate camera frame to

world need to rotate, translate, and scale

coordinate systems need to project world points to the

image plane all the above are modeled using

4x4 matrices and 1x4 points in homogeneous coordinates


14

Translation of 3D point PparametersPoint in 3D Point in

frame 1

Point in frame 2


15

Scaling 3D point P


16

Rotation of P about the X-axis


17

Rotate P about the Y-axis


18

Rotate P about the Z-axis

Looks same as 2D rotation omitting row, col 4


19

Arbitrary rotation has orthonormal rows and columns


20

Example: camera relative to world


21

exercise verify that the 3 x 3 rotation matrix

is orthonormal by checking 6 dot products

invert the 3 x 3 rotation matrix invert the 4 x 4 matrix verify that the new matrix

transforms points correctly from C to W


22

Transformation “calculus”: notation accounts for transforms

TW

M

Denotes transformation

Origin frame M

Destination frame W

T transforms points from model frame to world frame. (Notation from John Craig, 1986)


23

Apply transformations to points

TM

WP

MP =W

Point in model coordinates

Point in world coordinates

Transformation from model to world coordinates (instance transformation)


24

Matrix algebra enables composition

Let M and N be 4 x 4 matrices and let P be a 4 x 1 point

M ( N P ) = ( M N ) P we can transform P using N and then

transform that by M, or we can multiply matrices M and N and then apply that to point P

matrix multiplication is associative (but not commutative)


25

Composing transformations

T (W

CT(

C

AT =W

A

Two transformations are composed to get one transformation that maps points from the world frame to the frame A

Parameters: rotation and translation

Projection parms.cancel


26

Deriving form of the camera matrix

We have already described what the camera matrix does and what

form it has; we now go through the steps to justify it


27

Viewing model points M

What’s in front of the camera?


28

Math for the stepsCamera C maps 3D points in world W to 2D pixels in image I


29

Perspective transformation: camera origin at the center of projection

This transformation uses same units in 3D as in the image plane


30

Perspective projection: camera origin in the real image plane


31

Rigid transformation for change of coordinate frame

3D coordinate frame of camera


32

Relate camera frame to world frame


33

Change scene units to pixels

To get into XV or GIMP image coordinates! This is a 2D to 2D transformation.


34

Final result a 3x4 camera matrix maps 2D image pixels to 3D rays maps 3D rays to 2D image pixels obtain matrix via calibration

(easy) obtain matrix via reasoning (hard) do camera calibration exercise


35

The camera model, or matrix, is 3 x 4 and maps a homogeneous point in the world to a homogeneous pixel in the image. The ‘1’ is used to model translation

MSU CSE 803 Fall 2008 Stockman1 CV: 3D sensing and calibration Coordinate system changes;...

Documents

Transcript of MSU CSE 803 Fall 2008 Stockman1 CV: 3D sensing and calibration Coordinate system changes;...