The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland
-
Upload
shad-schroeder -
Category
Documents
-
view
20 -
download
3
description
Transcript of The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland
![Page 1: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/1.jpg)
The perception of Shading and ReflectanceE.H. Adelson, A.P. Pentland
Presenter: Stefan Zickler
![Page 2: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/2.jpg)
The “Intrinsic Image”
the underlying physical properties of a scene.
Looking at a 2D image, what does its 3-dimensional source model look like?
![Page 3: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/3.jpg)
What makes an image?
A combination of three factors: Lighting Shading Reflectance
![Page 4: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/4.jpg)
Lighting
Variables: Number of light sources Intensity Position Distribution (Spot-light or Global)
![Page 5: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/5.jpg)
Reflectance
How a surface’s material changes the light: Color Absorbance Transparency Etc…
![Page 6: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/6.jpg)
Shading
A change to the angle of incidence of light based on the surface normal.
![Page 7: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/7.jpg)
a simple formulation of an image in terms of reflectance and shading
I(x,y) = r(x,y) s(x,y) r(x,y) is the reflectance image s(x,y) is the shading image / luminance
image where s(x,y) = λ N(x,y)·L
N(x,y) is the surface normal L is the illumination direction λ is the “luminous flux”, meaning intensity of
light.
![Page 8: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/8.jpg)
The bad news
Any 2D image can be described by infinitely many 3D models of shading and reflectance (the most simple being a flat 2D screen, colored with the image).
![Page 9: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/9.jpg)
The good news
Humans are easily able to reason about which intrinsic 3D model is likely to be the correct one.
Therefore, a computer should be able do the same…
![Page 10: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/10.jpg)
How do we find the best intrinsic image?
A perception should correspond to the simplest or likeliest explanation.
One way to define simplicity is by introducing a cost-function.
![Page 11: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/11.jpg)
The “workshop” metaphor
A generative model for shading, reflectance, and lighting.
We have three workers: Painter Sheet Metal Worker Lighting Designer
![Page 12: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/12.jpg)
The painter
Can paint polygons with certain colors.
Works on the reflectance component of our image.
![Page 13: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/13.jpg)
The metal-worker
Can cut out new pieces of metal Can bend pieces of metal This is the shading component of
our image.
![Page 14: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/14.jpg)
The Lighting Designer
Can position lights to illuminate a scene.
Can chose between flood lights and spot lights.
![Page 15: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/15.jpg)
What does this give us?
A fairly complete generative model to create any arbitrary 3D scene
How do we enforce simplistic solutions? Through a cost-function.
![Page 16: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/16.jpg)
The pricelist
Painter Fees: Paint rectangular patch: $5 each Paint general polygon: $5 each
Sheet Metal Worker Fees: Right angle cuts $2 each Odd angle cuts $5 each Right angle bends $2 each Odd angle bends $5 each
Lighting Designer Fees: Flood light $5 each Custom spot light $30 each
![Page 17: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/17.jpg)
Painter’s solution: Paint 9 polygons: $180 Setup 1 flood light $5 Cut 1 rectangle $8 Total $193
Sheet metal worker's solution: Cut 24 odd angles $120 Bend 6 odd angles $30 Set up 1 flood light $5 Total $155
Lighting Designer's solution: Cut 1 Rectangle $8 Set up 9 spot lights $270 Total $278
Each worker can create an entire image with a minimum of help from the other workers.
![Page 18: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/18.jpg)
We need a supervisor
His role: Coordinate the three workers to find a
cooperative solution with the minimum overall cost.
In more scientific terms: To perform a search through the entire
solution space and find the point of minimum overall cost.
![Page 19: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/19.jpg)
The supervisor’s solution:
Supervisor's solution: Cut 1 rectangle $8 Paint 3 rectangles $5 Bend 2 right angles $4 Supervisor's fee $30 Total $47
Compare to: Painter’s solution: $193 Metal Worker’s solution: $155 Lighting Worker’s solution: $278
![Page 20: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/20.jpg)
Tweaking the price-list:Discouraging naïve solutions
Make naïve solutions expensive. We don’t want our algorithm to
simply create a painted 2D screen. On the other hand we don’t want to
make things like paint too expensive so that they never get used.
Cooperative solutions should be cheaper than single workers
![Page 21: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/21.jpg)
Is there an optimal pricelist?
Price-list values can be determined experimentally and tweaked in a way that they deliver the most likely solution for most images.
However, there is no universal price list that correctly describes all possible images.
![Page 22: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/22.jpg)
The main problem with this workshop theory
The search space for cooperative solutions of our workers is enormous, as there are infinitely many ways of combining their skills
Even for small scenes, there exists no efficient search algorithm to solve this problem in a simultaneous fashion.
![Page 23: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/23.jpg)
Their solution
Instead of a simultaneous cooperative model, we use a simplified, multi-stage generative model.
Where have we seen this before?
![Page 24: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/24.jpg)
Stage 1: The Shape Specialist
Assumptions: image was made by orthographic projection. We are given the observed x,y coordinates of
all edges and vertices in the image. Operations:
We can move vertices among the z axis
![Page 25: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/25.jpg)
Shape Specialist Contd.
Simple solutions are enforced by assigning higher costs to non-right angles.
Compactness (shorter edges) and planarity (less angle-variance) are rewarded.
This cost-metric works for most figures, but not all of them.
![Page 26: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/26.jpg)
Stage 2: Lighting Specialist
Given the shape from the previous specialist, find the lighting direction that best explains the observed luminance variation in terms of shading.
This can be estimated linearly by solving for the light direction L of two connected surfaces:
I1 = r1 λ N1·LI2 = r2 λ N2·L
Where r(x,y) is an estimated average, and λ=1
![Page 27: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/27.jpg)
Stage 3: Reflectance specialist
Given the shape and lighting from the previous two specialists, explain any left-over differences by painting the surfaces.
![Page 28: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/28.jpg)
An example:
![Page 29: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/29.jpg)
The problem with this approach
Real world scenes don’t look like this:
![Page 30: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/30.jpg)
The problem with this approach
Instead, they look more like this:
![Page 31: The perception of Shading and Reflectance E.H. Adelson, A.P. Pentland](https://reader036.fdocuments.us/reader036/viewer/2022062719/568132aa550346895d995395/html5/thumbnails/31.jpg)
Some Other Shortcomings
Tuning the cost-factors is done manually. There will never be a single set of parameters that will correctly describe all scenes.
A psychologist’s approach to computer science: not much information on how far this approach can scale up to more complex scenes, not much work on coming up with a better search algorithm or parameter learning.
How well this approach works on random, real-world scenes is questionable.