Horizon Map Capture

H. Rushmeier, L. Balmelli, F. Bernardini

IBM TJ Watson Research Center

I. Why Capture Horizon Maps?II. Capture & Processing Under Ideal ConditionsIII. Dealing with Real World DataIV. Some Example Maps

OVERVIEW

General Problem:How do we capture and represent existing objects ?

I. Why Capture Horizon Maps?

Image maps used to represent object detail:

Textures: colors

View Dependent Textures: changing colors

Normals: relit details

Surface Light Fields: all light from object

I. Why Capture Horizon Maps?

geometry

+ map

I. Why Capture Horizon Maps?

Surface Light Fields

Normals

Maps

Advantages All effects included: fast rendering

Limited effects included: can’t render some effects

Disadvantages All effects included: limited editing

Limited effects included: easy to edit

Goal:

Add captured cast shadows to normals maps to represent more lighting effects

Retain ability to edit

Store data in a form that is fast to render

I. Why Capture Horizon Maps?

Attached versus Cast Shadows:

I. Why Capture Horizon Maps?

Efficient representation of cast shadows:Horizon Maps (Max ’88)hardware rendering (Sloan & Cohen, ’00, Kautz et al. ’00)

I. Why Capture Horizon Maps?

Simple Hardware Set Up

I. Why Capture Horizon Maps?

Use Photometric Stereo to Compute Normals

I. Why Capture Horizon Maps?

Discard lightest and darkest values at each pixel, solve:

Li dot N,p = Gi,p

L = Light source direction, light iN = Surface normal at pGi,p = Gray scale, image I, pixel p

Why not reconstruct surface from normals?

Discontinuities:

Outliers:

I. Why Capture Horizon Maps?

Results are by integration:Effect of one bad normal

spreads across image

Why not reconstruct surface from normals?

I. Why Capture Horizon Maps?

Error in L dot N smaller than error in height.

Why not reconstruct surface from normals?

I. Why Capture Horizon Maps?

I. Why Capture Horizon Maps?

captured shadows from reconstructed heights

1. Identify regions of cast shadow

II. Capture and Processing Under Ideal Conditions

2. Identify height of ridge casting the shadow

II. Capture and Processing Under Ideal Conditions

Ridge height exact for sharp bumps only

II. Capture and Processing Under Ideal Conditions

Differentiate between bumps and grooves

II. Capture and Processing Under Ideal Conditions

3. Compute horizon map for each pixel by marching in light direction for each pixel until ridge encountered

ridge height h known

find distance from pixel to ridge in

Compute angle

II. Capture and Processing Under Ideal Conditions

Correcting for Finite Light Locations

III. Dealing With Real World Data

Shadows aren’t really black – analyze histograms

III. Dealing With Real World Data

No spike at zero “shadow”

Computing ridge locations: Identifying types of shadow edges

III. Dealing With Real World Data

Jaggy edge can cause pixel parallel to be misclassified as the “end” of the shadow

Computing Ridge locations:Combining, smoothing data

III. Dealing With Real World Data

IV. Some Example Maps

IV. Some Example Maps

captured corrected for finite light

IV. Some Example Maps

relit normals cast shadows

IV. Some Example Maps

horizon map reconstructed image

IV. Some Example Maps

IV. Some Example Maps

IV. Some Example Maps

IV. Some Example Maps

Editing

layers

Except for horizon maps, no pixel contains lighting information from other pixels. New horizon maps can be generated from edited ridges

IV. Some Example Maps

colors normals ridges horizonmap

IV. Some Example Maps

copydelete

cyan

blue

yellow

IV. Some Example Maps

copydelete

cyan

blue

yellow

IV. Some Example Maps