04 - 3D Object Models

8/8/2019 04 - 3D Object Models

1/40

3D objectcategorization

Courtesy of Prof. Silvio Savarese (U. Michigan,Ann-Arbor)


2/40


3/40

King

Arthurandthe knightsoftheRoundTable.


4/40

D Object Categorization

. Chiu et al 07 , ., Hoiem et al 07 , . Yan et al 07

. Weber et al 00 . Schneiderman et al 01

Capel et al 02 & Johnson Herbert 99 . Thomas et al 06 , ., Kushal et al 07 , ,Savarese et al 07 08

, Bronstein et al 03 - . ,Ruiz Correa et al 03 Funkhouser et al 03 Bart et al 04


5/40

D Object CategorizationChallenges

- ow to model 3D shape variabil i ty?

--

-- ( )ow to model texture appearancevariabil i ty?------ ( )ow to link texture appearance acrossviews?-


6/40

-ulti viewmodels

D Object Categorization

ixture of 2D single iew models .

Weber et al 00 . Schneiderman et al 01 . Bart et al 04

ull 3D models , Bronstein et al 03 - . ,Ruiz Correa et al 03 Funkhouser et al 03 Capel et al 02 & Johnson Herbert 99

. Thomas et al 06 , ,Savarese et al 07 08 . Chiu et al 07 , ., Hoiem et al 07 , . Yan et al 07

, ., Kushal et al 07 Liebelt et al 08 , , ,Sun Su Savarese et al 09a09b


7/40

ingle 3D object recognition ingle view objectcategorization

D object categorization

Overview


8/40

Zhang et al 95 & , Schmid Mohr 96 & , Schiele Crowley 96 , Lowe 99 & , Jacob Barsi 99

. Edelman et al 91 & , Ullman Barsi 91 Rothwell 92 , Linderberg 94 & Murase Nayar 94

., Rothganger et al 04 , Ferrari et al 05 & Brown Lowe 05 Snavely et al 06 & , Yin Collins 07

, Ballard 81 & .- , Grimson L Perez 87

, Lowe 87

ingle 3D objectrecognit ion


9/40

.

C o u r t e s

y

o f

D

L o w e

C o u r t e s

y

o f

R o t h

g a n

g e r

e t

a l

Difference of Gaussian ( ):DOG , used in Lowe 99 Brown et al 05

- :Harris Laplace used in . Rothganger et al 06 :Laplacian used in . Lazebnik et al 04


10/40

:bject representat ion ollection of patches in 3D

. Rothganger et al 06

, , +x y z, +h v

descriptor

C o u

r t e s

y

o f

R o t h

g a n

g e r

e t

a l


11/40

odel learning . Rothganger et al 03 06

N images of object from N different view points

Match key points between consecutive views

[ ]create sample set

Use affine structure from motion to compute 3D location and

+orientation camera locations[ ]RANSAC

Upgrade model to Euclidean assuming zero skew and square pixels

Find connected components

Use bundle adjustment to refine model

:uild a 3D model


12/40


13/40

Recognit ion [ . ]Rothganger et al 03 06

.1 Find matches between model and test imagefeatures . :2 Generate hypothesis

Compute transformation M from N matches ( = ; ;N 2 affine camera affine)key points

.3 Model verification Use M to project other matched 3D model features into testimage


14/40

Recognit ion [ . ]Rothganger et al 03 06

:Goal( )Estimate fit the best M in presence ofoutliers


15/40

Multi-view models

3D Object Categorization

Mixture of 2D single viewmodels

Weber et al. 00 Schneiderman et al. 01 Bart et al. 04

Full 3D models Bronstein et al, 03 Ruiz-Correa et al. 03, Funkhouser et al 03 Capel et al 02 Johnson & Herbert 99

Thomas et al. 06 Savarese et al, 07, 08

Chiu et al. 07 Hoiem, et al., 07 Yan, et al. 07 Kushal, et al., 07 Liebelt et al 08 Sun et al 08


16/40

Full 3Dmodels

3D modelinstance

3D modelinstance

3D category

model

Bronstein et al, 03 Ruiz-Correa et al. 03, Funkhouser et al 03 Kazhdan et al.03 Osada et al 02 Capel et al 02 Johnson & Herbert 99 Amberg et al 08

A 3D model category is built from a collection of 3D range data or CAD models


17/40

Osada et al 02Shape distributions

Kazhdan et al. 03Spherical harmonics


18/40

Full 3Dmodels

3D modelinstance

3D modelinstance

3D category

model

-Build a 3d model is expensive-Difficult to incorporate appearance information- Need to cope with 3D alignment (orientation, scale, etc)

Bronstein et al, 03 Ruiz-Correa et al. 03, Funkhouser et al 03 Kazhdan et al.03 Osada et al 02 Capel et al 02 Johnson & Herbert 99 Amberg et al 08

A 3D model category is built from a collection of 3D range data or CAD models


19/40

Multi-view models

3D Object Categorization

Mixture of 2D single viewmodels

Weber et al. 00 Schneiderman et al. 01 Bart et al. 04

Full 3D models Bronstein et al, 03 Ruiz-Correa et al. 03, Funkhouser et al 03 Capel et al 02 Johnson & Herbert 99

Thomas et al. 06 Savarese et al, 07, 08

Chiu et al. 07 Hoiem, et al., 07 Yan, et al. 07 Kushal, et al., 07 Liebelt et al 08 Sun et al 08


20/40


21/40

Yan, et al. 07

Multi-view models by rough 3d shapes


22/40

Hoiem, et al., 07

Multi-view models by rough 3d shapes


23/40

C

ourtesyofThomasetal.06

[Thomas et al. 06]

Multi-view models by ISM representations


24/40

Sparse set of interest points or parts of the objects are linked across views.

Multi-viewmodel

[Thomas et al. 06]

Multi-view models by ISM representations


25/40

A nified framework for 3D object, ,detection pose classification pose

synthesis, - ,avarese Fei Fei ICCV 07

, - ,avarese Fei Fei ECCV 08, , , - ,un Su Savarese Fei Fei CVPR 09

, , - , ,u Sun Fei Fei Savarese ICCV 09

anonical parts captures diagnostic appearanceinformation

d structure linking parts via weak geometry


26/40

Canonical parts

,ysical par t is planar canonical part is stable point on the manifo anonical part can be computed from connected component ofparts

onnected

omponent of parts


27/40

Canonical parts

onnected omponent of parts

k


28/40

Linkage structure

Obj R i i


29/40

Object Recognition

. Find hypotheses of canonical parts consistent with a iven pose

uery image

model

Algorithm

Obj R i i


30/40

Object Recognition

. Find hypotheses of canonical parts consistent with a iven pose

uery image

. Infer posit ion and pose of other canonical parts

model

Algorithm

Obj t R iti


31/40

Object Recognition uery image

model

. Find hypotheses of canonical parts consistent with given pose .

Infer posit ion and pose of other canonical parts

Algorithm

Obj t R iti


32/40

Object Recognition uery image

. Optimize over E , G nd s o find best combination ofhypothesis

model

. Find hypotheses of canonical parts consistent with a iven pose . Infer posit ion and pose of other canonical parts

Algorithm

error

ifi d f k


33/40


synthesis, - ,

avarese Fei Fei ICCV 07



-robabil istic generative part based model

-ense Multi view representation on the viewing sphere

ifi d f k f b


34/40


synthesis, - ,

avarese Fei Fei ICCV 07



: , ,0 50am Thursday Oct 1 Oral session 9

. *, . *, . - , . ,Su M Sun L Fei Fei S Savarese Learning

- ,a Dense Multi view Representation for Detection Viewpoint Classification and Synthesis of Object.Categories

f d f k f b


35/40

A unified framework for 3D objectdetection , ,pose classification pose

synthesis


A ifi d f k f 3D bj


36/40

A unified framework for 3D object,detection ose classification ,pose synthesis


A ifi d f k f 3D bj


37/40



A ifi d f k f 3D bj


38/40



A ifi d f k f 3D bj


39/40

A unified framework for 3D object, ,detection pose classification ose

ynthesis


A ifi d f k f 3D bj t


40/40

A unified framework for 3D object, ,detection pose classification ose

ynthesis Single

view/Mixture

-Multiview

. ,Sav et al 07

Morphing model ( ,Su

, . )Sun et al 09 View point

invariant X No supervision X X all views all instances

available

# Categories ~ 300 2 8 16 Share information across views X

View synthesis X X X Pose estimation X

View point

04 - 3D Object Models

Documents

Transcript of 04 - 3D Object Models