Introduction

Adrien Bartoli

ALCoV – ISIT

Université d’Auvergne

Clermont-Ferrand, France

Fourth Tutorial on Computer Vision in a Nonrigid World ICCV’11, Barcelona, Spain – octobre 6, 2011

Lourdes Agapito, Adrien Bartoli, Alessio Del Bue

Millennia

Century

Decades

“Vision is the act of knowing what is where by looking.”

- Aristotle

Object detection Person detection Person recognition

Tob

y

Ad

rien

Ab

ed

Jae-Hak

Florian

Image Understanding in Computer Vision

Measurements

“Vision is the act of knowing what is where by looking.”

- Aristotle

Measurements

3D Computer Vision

Depth map Partial 3D surface

Real world

A General Sketch of Visual Process

Tob

y

Ad

rien

Ab

ed

Jae-Hak

Florian

Image Formation

Shape-from-X: Inversing Image Formation

?

The X of Shape-from-X

Shading Focus Shadow

Texture Shading+specularity

Silhouette Occlusion

… and more!

Scene Priors

Shape priors – often statistical

→ A kettle is rigid

→ Pretrained models: I saw faces before, I learnt their shape

Deformation priors – often physical

→ Untrained models: the shape is structured

→ A piece of paper is isometric and developable

→ A face is quasi-isometric

= 𝑓( ) From [Blanz and Vetter, PAMI’03]

= 𝑓( )

Real world

A Sketch of the Current Visual Process in SfX

Tob

y

Ad

rien

Ab

ed

Jae-Hak

Florian

Using Multiple-Views

Photometric stereo

Shape-from-Motion - SfM

Image formation

Stereovision Monovision

Are Multiple Views Always Enough?

The answer depends on the scene prior

Rigid Deformable

In general, yes No general answer

State of research: mature State of research: open

Rigid SfM has Two Main Steps

Image registration Shape inference

Registered images with matched keypoints

Non-Rigid SfM / Deformable SfM

Image registration Shape inference

Registered images with visualization grid

Template-Based Deformable 3D Reconstruction

Image

registration

template

Shape

inference

Tutorial Roadmap

• Introduction

1. Deformable Image Registration – Feature-based – Pixel-based

2. Template-Based Deformable Shape Inference – Isometric surfaces – Elastic surfaces

3. Non-Rigid Structure-from-Motion – The Low-Rank Shape Model – Alternative Models

• Conclusion

+

… …

Mo

rnin

g M

orn

ing

Aft

ern

oo

n

Introduction

Adrien Bartoli

ALCoV – ISIT

Université d’Auvergne

Clermont-Ferrand, France

Fourth Tutorial on Computer Vision in a Nonrigid World ICCV’11, Barcelona, Spain – octobre 6, 2011

Lourdes Agapito, Adrien Bartoli, Alessio Del Bue