Direct Linear Transformation & Computer Vision Models · CE 59700: Digital Photogrammetric Systems...

CE 59700: Digital Photogrammetric Systems Ayman F. Habib1

Direct Linear Transformation & Computer Vision Models

Chapter 7-A4

Photogrammetry Vs. Computer Vision• Conventional Photogrammetry is focusing on precise

geometric information extraction from imagery.– Topographic mapping from space borne and airborne imagery– Metrological information extraction through close-range

photogrammetry (terrestrial photogrammetry)• Object-to-camera distance is less than 100meter

• Computer Vision (CV) is mainly concerned with automated image understanding:– Object recognition,– Navigation and obstacle avoidance, and– Object modeling

Airborne Photogrammetric Mapping

Close-Range Photogrammetric Mapping

CV: Object Recognition

CV: Navigation & Obstacle Avoidance

Photogrammetry Vs. Computer Vision• Photogrammetry is always concerned with precise

geometric information extraction.– Photogrammetric mapping considers potential deviations from

the assumed perspective projection.

• For Computer Vision (CV):– Focus is always on automation.– Object recognition and navigation applications do not require

precise derivation of geometric information.– Depending on the application, object modeling might require

precise geometric information extraction.– CV usually assumes that the collinearity of the object point,

perspective center, and corresponding image point is maintained, even for un-calibrated cameras.

Object-to-Image Coordinate Transformation in Photogrammetry

Collinearity Equations

oa = oA

These vectors should be defined w.r.t.the same coordinate system.

(xa, ya)

R( , , )

(Perspective Center)

The vector connecting the perspective center to the image point

w.r.t. the image coordinate system

distyydistxx

distydistx

rv ypa

The vector connecting the perspective center to the object point

w.r.t. the ground coordinate system

ZZYYXX

Where: is a scale factor (+ve).

11 12 13

21 22 23

31 32 33

( , , )c c mi oa O m oA

a p x A o

a p y A o

v r M V R rx x dist m m m X Xy y dist m m m Y Y

c m m m Z Z

Collinearity EquationscmRM

yoAoAoA

oAoAoApa

xoAoAoA

oAoAoApa

distZZmYYmXXmZZmYYmXXmcyy

distZZmYYmXXmZZmYYmXXmcxx

)()()()()()()()()()()()(

333231

232221

333231

131211

yoAoAoA

oAoAoApa

xoAoAoA

oAoAoApa

distZZrYYrXXrZZrYYrXXrcyy

distZZrYYrXXrZZrYYrXXrcxx

)()()()()()()()()()()()(

332313

322212

332313

312111

Object-to-Image Coordinate Transformation

Direct Linear TransformationComputer Vision Model

DLT & Computer Vision Models• The DLT and computer vision models encompass:

– Collinearity Equations,– Non-orthogonality () between the axes of the image/camera

coordinate system, and– Two scale factors (Sx, Sy) along the axes of the image

coordinate system.

• DLT & CV models can directly deal with pixel coordinates.

• We will start with modifying the rotation matrix to consider the impact of the non-orthogonality ().– Primary rotation @ the -axis of the ground coord. system– Secondary rotation @ the -axis– Tertiary rotation & @ the -axis

Primary Rotation ()

cossin0sincos0001

Secondary Rotation ()

cos0sin010

sin0cos

Tertiary Rotation ()

1000cossin0sincos

Rotation in Space

RRRzyx

// to the ground coordinate system // to the image coordinate system

Rotation in Space

coscossinsincoscossin

cossincossinsincossin

sinsinsincoscoscossinsinsincos

sinsincos

coscos:

333231

232221

131211

rrrrrrrrrwhere

rrrrrrrrr

Consideration of the Non-Orthogonality ()

1000)cos(sin0)sin(cos

1000sincossin0cossincos

sincossinsincoscos)cos(cossinsincoscossin)sin(

Assuming small non-orthogonality angle ()

10001001

1000cossin0sincos

1000sincossin0cossincos

RRRZYX

10001001

// to the image coordinate system // to the ground coordinate system

10001001

Note: 1 00 1 00 0 1

1 00 1 00 0 1

Consideration of the Non-Orthogonality ()• Collinearity Equations while considering the non-

orthogonality () between the axes of the image coordinate system.

ZZYYXX

Rcyyxx

10001001

ZZYYXX

syysxx

10001001

/)(/)(

ZZYYXX

Rccsyycsxx

10001001

)/()()/()(

• Divide both sides by (-c).

Consideration of the Scale Factors• Collinearity Equations while considering the non-

orthogonality () between the axes of the image coordinate system & different scale factors.

Consideration of the Scale Factors

ZZYYXX

Rcyycxx

10001001

1)/()()/()(

ZZYYXX

10001001

1000/1000/1

ZZYYXX

10001001

1000000

• csx→ cx , csy→ cy & -/c → `.

DLT & Computer Vision Models

ZZYYXX

100000

ZZYYXX

rrrrrrrrr

332313

322212

312111

100000

Where:

0 { }0 0 1

Yy K R I X

c c xK c y Calibration Matrix

R I X Exterior Orientation Matrix

DLT & Computer Vision Models• The Direct Linear Transformation (DLT), which has

been developed by the photogrammetric community, is an alternative to the collinearity equations that allows for direct transformation between machine/pixel coordinates and corresponding ground coordinates.– &

• The DLT can be also represented by the following form:

–1 1

9 10 11

' 00 0 1

x x pT

L L L c c xD L L L c y R

' 00 0 1

x x p OT

L c c x XL c y R YL Z

DLT: Direct Linear Transformation

DLT & CV Models: Pixel Coordinates• The DLT & CV models can also consider the direct

transformation from pixel to ground coordinates.

⁄ _ _⁄ _ _

DLT & CV Models: Pixel Coordinates⁄ _ _

⁄ _ _

_ _ ⁄ _ __ _ ⁄ _ _

DLT & CV Models: Pixel Coordinates_ _ ⁄ _ _

_ _ ⁄ _ _

_ _ ⁄ _ __ _ ⁄ _ _

_ _⁄ ⁄_ _⁄ ⁄

DLT & CV Models: Pixel Coordinates• Modified Calibration Matrix:

_ _⁄ ⁄_ _⁄ ⁄

_ _⁄ _ _⁄ _ _⁄ ⁄0 _ _⁄ _ _⁄ ⁄0 0 1

DLT & CV Models: Pixel Coordinates• For DLT when working with pixel coordinates, we have

the following model.

Modern Photogrammetry & Computer Vision• Modern Photogrammetry and Computer Vision are

converging fields.

Art and science of tool development for automatic generation of spatial and descriptive information from

multi-sensory data and/or systems

DLT → IOPs & EOPs

Approach # 1

DLT → IOP & EOP

9 10 11

00 0 1

x x pT

L L L c c xD L L L c y R

00 0 1

x x p OT

L c c x XL c y R YL Z

DLT → IOP & EOP

No Sign Ambiguity

LLLLLLLLL

• Given:

• Then:

LLLLLLLLL

DLT → IOP & EOP

cycxcc

3212)()(

LLLLLLLLL

KKRKRKDD TTTTT

2933)( LLLDD T

29 AmbiguitySignLLL

DLT → IOP & EOP

pT xLLLLLLDD 2

3112101913)(

31121019LLL

LLLLLLxp

No Sign Ambiguity

3212)()(

LLLLLLLLL

KKRKRKDD TTTTT

pT yLLLLLLDD 2

7116105923)(

DLT → IOP & EOP

71161059LLL

LLLLLLyp

No Sign Ambiguity

3212)()(

LLLLLLLLL

KKRKRKDD TTTTT

)()( 22227

2522 yp

T cyLLLDD

DLT → IOP & EOP

3212)()(

LLLLLLLLL

KKRKRKDD TTTTT

py yLLL

No Sign Ambiguity

DLT → IOP & EOP

)()( 273625121 ppyx

T yxccLLLLLLDD

3212)()(

LLLLLLLLL

KKRKRKDD TTTTT

ppyx yxLLL

LLLLLLcc )(/1 211

736251

No Sign Ambiguity

)()( 2222223

2111 pxx

T xccLLLDD

DLT → IOP & EOP

3212)()(

LLLLLLLLL

KKRKRKDD TTTTT

5.0222

pxx xcLLL

No Sign Ambiguity

DLT → IOP & EOP• Given:

• Then: sin139 rL

Sign Ambiguity

332313

322212

312111

rrrrrrrrr

LLLLLLLLL

Collinearity Equations• Objective: Resolve the sign ambiguity in

• Since the scale factor is always +ve

• Assuming that the origin (0, 0, 0) is visible in the imagery

veZZrYYrXXr OOO )()()( 332313

veZrYrXr OOO 332313

)()()()()()()()()(

332313

322212

312111

ZZrYYrXXrZZrYYrXXrZZrYYrXXr

Scyyxx

DLT → IOP & EOP

• By choosing L12 = 1.

12 13 23 33

13 23 33

( )1 ( )

L r X r Y r Zr X r Y r Z

r X r Y r Z

is Negative

29 LLL

Rycxcc

DLT → IOP & EOP

• No sign Ambiguity

LLLLrL

DLT → IOP & EOP

coscoscossin

10tan LL

No Sign Ambiguity

332313

322212

312111

rrrrrrrrr

LLLLLLLLL

DLT → IOP & EOP

333231

232221

131211

332313

322212

312111

LLLLLLLLL

rrrrrrrrr

LLLLLLLLL

• Retrieve

• Note: There is an ambiguity in determination ( cannot be distinguished).

coscos 11r

DLT → IOP & EOP

Approach # 2: Matrix Factorization

DLT → IOP (Factorization # 1)• Conceptual basis: Direct derivation of the calibration matrix

• Cholesky Decomposition of DDT→ K (Calibration Matrix)? Wrong

cycxcc

LLLLLLLLL

KKRKRKDD

NMMMNCHO

TDDN TKK

11)]}({[ TDDCHOK

DLT → IOP (Factorization # 2)

333231

232221

131211

332313

322212

312111

LLLLLLLLL

rrrrrrrrr

LLLLLLLLL

• Using the rotation matrix R, one can derive the individual rotation angles , and .

DLT → Rotation Angles

Analysis

Perspective Center

1211109

LZLYLXLLZLYLXLLZLYLXL

LLLLLLLLL

• (XO, YO , ZO) is the intersection point of three different planes whose surface normals are (L1, L2, L3), (L5, L6, L7) and (L9, L10, L11), respectively.

Perspective Center

332313

322212

312111

rrrrrrrrr

LLLLLLLLL

• Assuming:– xp ≈ 0.0 and yp ≈ 0.0– -cx ≈ 0.0

332313

322212

312111

rrrrcrcrcrcrcrc

LLLLLLLLL

• The three surfaces are orthogonal to each other.– This would lead to better intersection.

332313

322212

312111

rrrrrrrrr

LLLLLLLLL

• Assuming:– xp ≠ 0.0 and yp ≠ 0.0– -cx ≈ 0.0

332313

333223221312

333123211311

rrrryrcryrcryrcrxrcrxrcrxrc

LLLLLLLLL

pypypy

pxpxpx

• As xp and yp increase, the surface normals become almost parallel.– This would lead to weak intersection.

Perspective Center

• The rows of D are not correlated:– They are orthogonal to each other.

• L-1 is well defined.

333231

232221

131211

LLLLLLLLL

rrrrrrrrr

• Assuming:– xp ≈ 0.0 and yp ≈ 0.0– -cx ≈ 0.0

332313

322212

312111

rrrrcrcrcrcrcrc

LLLLLLLLL

Rotation Angles

333231

232221

131211

LLLLLLLLL

rrrrrrrrr

• Assuming:– xp ≠ 0.0 and yp ≠ 0.0– -cx ≈ 0.0

332313

333223221312

333123211311

rrrryrcryrcryrcrxrcrxrcrxrc

LLLLLLLLL

pypypy

pxpxpx

• The rows of D tend to be highly correlated.• L-1 is not well defined.

Rotation Angles

Direct Linear Transformation & Computer Vision Models · CE 59700: Digital Photogrammetric Systems...

Documents

Transcript of Direct Linear Transformation & Computer Vision Models · CE 59700: Digital Photogrammetric Systems...

PHOTOGRAMMETRY baselining

SYMPOSIUM NON-TOPOGRAPHIC PHOTOGRAMMETRY ......Non-Topographic Photogrammetry Author Bertil Hallert Subject Non-Topographic Photogrammetry Keywords Photogrammetric Engineering & Remote

TOWARDS AUTONOMOUS MARS ROVER LOCALIZATION: OPERATIONS … · KEY WORDS: Absolute Orientation, Bundle Adjustment, Close Range Photogrammetry, Computer Vision, Matching, Navigation

Contribution of the Department of Surveyng fileDevelopment in the field of machine vision, robot vision and computer vision and fully automated photogrammetric systems (real time photogrammetry)

CE 59700: Digital Photogrammetric Systems...CE 59700: Digital Photogrammetric Systems 4 Ayman F. Habib Course Notes and Textbooks • Material presented in class, as well as supplemental

PhotoGRammetry · photogrammetry: aerial, terrestrial, and close ranger. ... what is a stereoplotter?

Introduction to Photogrammetry - GIS Resources · LPS Project Manager Introduction / 15 Introduction to Photogrammetry Introduction What is Photogrammetry? Photogrammetry is the “art,

computer vision photogrammetry for underwater archaeological site ...

Photogrammetry article

Photogrammetry 2003

UNIT 1 : PHOTOGRAMMETRY · Unit 1 Photogrammetry Photogrammetry Page 3 1.1 INTRODUCTION 'Photo' - light’ Gram' – drawing'Metry' – measurement Simply stated, Photogrammetry is

Photogrammetry chandu

Geospatial World Forum Jan 19-21,2011 1 Future of Photogrammetry Rolta’s Vision.

CLOSE RANGE PHOTOGRAMMETRY AND MACHINE VISION

Introduction to Photogrammetry - GIS Resources Project Manager Introduction / 15 Introduction to Photogrammetry Introduction What is Photogrammetry? Photogrammetry is the …

PHOTOGRAMMETRY - Keith W. Spencerkeithwspencer.net/LSReview/Caltrans/LS-LSIT-Unit09_Photogrammet… · Introduction The Manual of Photogrammetry defines photogrammetry as, “the

Photogrammetry- Surveying

Photogrammetry Caltrans

Photogrammetry amir

PHOTOGRAMMETRY - Gancell · Photogrammetry Basics of Photogrammetry Before describing the operation of the V-STARS system, a brief introduction to photogrammetry is provided for those