Computational Vision

34
Computational Vision Jitendra Malik University of California, Berkeley

description

Computational Vision. Jitendra Malik University of California, Berkeley. What is in an image?. The input is just an array of brightness values; humans perceive structure in it. Water. back. Grass. Tiger. Tiger. Sand. head. eye. legs. tail. mouse. shadow. From Pixels to Perception. - PowerPoint PPT Presentation

Transcript of Computational Vision

Page 1: Computational Vision

Computational Vision

Jitendra MalikUniversity of California, Berkeley

Page 2: Computational Vision

What is in an image?

The input is just an array of brightness values; humans perceive

structure in it.

Page 3: Computational Vision

From Pixels to Perception

TigerGrass

Water

Sand

outdoorwildlife

Tiger

tail

eye

legs

head

back

shadow

mouse

Page 4: Computational Vision

If visual processing was purely feedforward…(it isn’t)

Pixels Local Neighborhoods

Contours Surfaces

TigerGrass

Water

Sand

ObjectsScenes

Low-level

Image Processing

Mid-level

GroupingFigure/Ground

Surface Attributes

High-level

Recognition

Page 5: Computational Vision

Boundaries of image regions defined by a number of attributes

Brightness/color Texture Motion Binocular disparity Familiar configuration

Page 6: Computational Vision
Page 7: Computational Vision
Page 8: Computational Vision

Grouping is hierarchicalA

B C

• A,C are refinements of B• A,C are mutual refinements • A,B,C represent the same

percept

Image

BG L-bird R-bird

grass bush

headeye

beakfar body

headeye

beak body

Perceptual organization forms a tree:

Two segmentations are consistent when they can beexplained by the samesegmentation tree

Page 9: Computational Vision

Humans assign a depth ordering to surfaces across a contour

R1 appears in front of R2 R2 appears in front of R3

This can be done for images of natural scenes …

Page 10: Computational Vision

Figure-Ground Labeling

-

- red is near; blue is far

Page 11: Computational Vision

Figure/Ground Organization

A contour belongs to one of the two (but not both) abutting regions.

Figure(face)

Ground(shapeless)

Figure(Goblet)Ground

(Shapeless)

Important for the perception of shape

Page 12: Computational Vision

Some other aspects of perceptual organization

Good continuation Amodal completion Modal completion

Page 13: Computational Vision

What do we see here?

Page 14: Computational Vision

And here?

Page 15: Computational Vision

Some Pictorial Cues

Page 16: Computational Vision

Support, Size

?

??1

3

2

Page 17: Computational Vision

Cast Shadows

Page 18: Computational Vision

Shading

Page 19: Computational Vision

Measuring Surface Orientation

Page 20: Computational Vision

Binocular Stereopsis

Page 21: Computational Vision

Optical flow for a pilot

Page 22: Computational Vision

Object Category Recognition

Page 23: Computational Vision

Shape variation within a category

D’Arcy Thompson: On Growth and Form, 1917 studied transformations between shapes of organisms

Page 24: Computational Vision

Attneave’s Cat (1954)Line drawings convey most of the information

Page 25: Computational Vision

Objects are in Scenes

Page 26: Computational Vision

Human stick figure from single image

Input image Stick figure Support masks

Page 27: Computational Vision

This is hard…

Variety of poses Clothing Missing parts Small support for parts Background clutter

Page 28: Computational Vision

Taxonomy and Partonomy Taxonomy: E.g. Cats are in the order Felidae which in

turn is in the class Mammalia Recognition can be at multiple levels of categorization, or be

identification at the level of specific individuals , as in faces. Partonomy: Objects have parts, they have subparts

and so on. The human body contains the head, which in turn contains the eyes.

These notions apply equally well to scenes and to activities.

Psychologists have argued that there is a “basic-level” at which categorization is fastest (Eleanor Rosch et al).

In a partonomy each level contributes useful information for recognition.

Page 29: Computational Vision

Visual Control of Action

Locomotion Navigation/Way-finding Obstacle Avoidance

Manipulation Grasping Pick and Place Tool use

Page 30: Computational Vision
Page 31: Computational Vision

Camera Obscura(Reinerus Gemma-Frisius, 1544)

Page 32: Computational Vision

Camera Obscura(Angelo Sala, 1576-1637)

Page 33: Computational Vision
Page 34: Computational Vision