Computational Vision
description
Transcript of Computational Vision
![Page 1: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/1.jpg)
Computational Vision
Jitendra MalikUniversity of California, Berkeley
![Page 2: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/2.jpg)
What is in an image?
The input is just an array of brightness values; humans perceive
structure in it.
![Page 3: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/3.jpg)
From Pixels to Perception
TigerGrass
Water
Sand
outdoorwildlife
Tiger
tail
eye
legs
head
back
shadow
mouse
![Page 4: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/4.jpg)
If visual processing was purely feedforward…(it isn’t)
Pixels Local Neighborhoods
Contours Surfaces
TigerGrass
Water
Sand
ObjectsScenes
Low-level
Image Processing
Mid-level
GroupingFigure/Ground
Surface Attributes
High-level
Recognition
![Page 5: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/5.jpg)
Boundaries of image regions defined by a number of attributes
Brightness/color Texture Motion Binocular disparity Familiar configuration
![Page 6: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/6.jpg)
![Page 7: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/7.jpg)
![Page 8: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/8.jpg)
Grouping is hierarchicalA
B C
• A,C are refinements of B• A,C are mutual refinements • A,B,C represent the same
percept
Image
BG L-bird R-bird
grass bush
headeye
beakfar body
headeye
beak body
Perceptual organization forms a tree:
Two segmentations are consistent when they can beexplained by the samesegmentation tree
![Page 9: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/9.jpg)
Humans assign a depth ordering to surfaces across a contour
R1 appears in front of R2 R2 appears in front of R3
This can be done for images of natural scenes …
![Page 10: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/10.jpg)
Figure-Ground Labeling
-
- red is near; blue is far
![Page 11: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/11.jpg)
Figure/Ground Organization
A contour belongs to one of the two (but not both) abutting regions.
Figure(face)
Ground(shapeless)
Figure(Goblet)Ground
(Shapeless)
Important for the perception of shape
![Page 12: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/12.jpg)
Some other aspects of perceptual organization
Good continuation Amodal completion Modal completion
![Page 13: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/13.jpg)
What do we see here?
![Page 14: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/14.jpg)
And here?
![Page 15: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/15.jpg)
Some Pictorial Cues
![Page 16: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/16.jpg)
Support, Size
?
??1
3
2
![Page 17: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/17.jpg)
Cast Shadows
![Page 18: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/18.jpg)
Shading
![Page 19: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/19.jpg)
Measuring Surface Orientation
![Page 20: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/20.jpg)
Binocular Stereopsis
![Page 21: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/21.jpg)
Optical flow for a pilot
![Page 22: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/22.jpg)
Object Category Recognition
![Page 23: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/23.jpg)
Shape variation within a category
D’Arcy Thompson: On Growth and Form, 1917 studied transformations between shapes of organisms
![Page 24: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/24.jpg)
Attneave’s Cat (1954)Line drawings convey most of the information
![Page 25: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/25.jpg)
Objects are in Scenes
![Page 26: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/26.jpg)
Human stick figure from single image
Input image Stick figure Support masks
![Page 27: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/27.jpg)
This is hard…
Variety of poses Clothing Missing parts Small support for parts Background clutter
![Page 28: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/28.jpg)
Taxonomy and Partonomy Taxonomy: E.g. Cats are in the order Felidae which in
turn is in the class Mammalia Recognition can be at multiple levels of categorization, or be
identification at the level of specific individuals , as in faces. Partonomy: Objects have parts, they have subparts
and so on. The human body contains the head, which in turn contains the eyes.
These notions apply equally well to scenes and to activities.
Psychologists have argued that there is a “basic-level” at which categorization is fastest (Eleanor Rosch et al).
In a partonomy each level contributes useful information for recognition.
![Page 29: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/29.jpg)
Visual Control of Action
Locomotion Navigation/Way-finding Obstacle Avoidance
Manipulation Grasping Pick and Place Tool use
![Page 30: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/30.jpg)
![Page 31: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/31.jpg)
Camera Obscura(Reinerus Gemma-Frisius, 1544)
![Page 32: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/32.jpg)
Camera Obscura(Angelo Sala, 1576-1637)
![Page 33: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/33.jpg)
![Page 34: Computational Vision](https://reader035.fdocuments.us/reader035/viewer/2022062500/56814fbb550346895dbd7456/html5/thumbnails/34.jpg)