Shading-Based Surface Editing

Yotam Gingold and Denis Zorin

New York University

SIGGRAPH 08

Abstract A free-form surface modeling based on

shading

Outline1. Introduction2. Related Work3. Shading Changed to Shape Changes4. Overview of the System5. Problem Formulation6. Results7. Conclusions and Future Work

Introduction

Surface Editing 2D UI 3D model & motion Shape-from-shading (SfS) reconstruction Sketch-based modeling

Motivation Indirection

User action vs. appearance change Hard to deform

Smooth outline Remove shadow Reshape a highlight

Purpose A directly sketched-based surface modeling Principle of continuity

if a user makes a small change in surface appearance, the resulting shape change should be small

Challenges

standard formulation (Lambertian surface, orthographic projection, directional light) is known to be ill-posed.

Challenges Avoid small shading modification leading to

large and unintuitive model changes

Preserve existing surface detail during editing

Region of interesting (ROI) during modifying

Realtime surface update

Our major techniques 1. Design stroke-based 2D UI2. SfS by solving a quadratic optimization

Related Work

Related Work Shape-Preserving

[Sorkine et al. 04] [Yu et al. 04] [Wardetzky et al. 07]

Shape-from-Shading[Rushmeier et al. 03][Prados 04]

Sketch-based modeling[Igarashi et al. 99]

[Cheutet et al. 04]

[Lawrence and Funkhouser 04]

[Kara et al. 06]

[Karpenko and Huges 06]

[Nealen et al. 07]

Silhouette Editing[DeCarlo et al. 03] . Suggestive contour

[Nealen et al. 05],

[Zimmermann et al. 07]

Shading Changes to Shape Changes

Guarantee the stability of surface changes and satisfying boundary constraints

a continuous solution an approximate solution

all solutions are discontinuous

(with either one or two sides fixed)

Instability near highlights Conclusion

Smooth deformation can’t erase highlight

A large change in the surface shape

Strategy Terminate erasing

strokes at highlight

Highlight removal

Slope ambiguity

Convex-concave ambiguity slope ambiguity

Strategy - choose the slope to change the surface the least

Overview of the System

M surface

pvpl

vL

Lambertian & glossy

reflection model , n(p)

I

~I~M surface

q

p

q

I(q) = ρ(n(p))

User modified

I image

n

Only one light source

Lambertian and Glossy Reflection Model

β is the degree of glossiness p is the Phong exponent h = (v+l) / |v+l|

M surface

pl

v

LLambertian &

glossy reflection model ,

n(p)

~I

q

p

q

I(q) = ρ(n(p))

I image

n

C

P-1(C)

q = P(p)

Only one light source

brush

attributes of brush

Shading modification brush

Silhouette brush Highlight motion brush ROI pen

• Opacity• Smoothness• Width

Stroke attributes – α (opacity)

Replace mode Multiply mode

0)1(I IIvtrq 00 )1(),1min(I IIIvtrq

Stroke attributes – f (softness)

Stroke attributes – w (width)

Problem Formulation

Surface Optimization Function Detail-preserving

Preserving appearance outside strokes Stroke constrain

Match the modified surface under the stroke

Detail-preserving Stroke constrain

Detail-preserving

[Yu et al. 04]

The vector Laplacian is the normal scaled by the mean curvature [Sorkine et al. 04]

If the surface changes remain close to isometric, the Laplacian operator does not change [Wardetzky et al. 07]. The Laplacian difference

ΔM : Laplace-Beltrami operatorH: mean curvature

Small triangle distortion ? = isometric deformations

Hypothesis If the triangle distortion stays small, one can

view the Laplacian difference energy as a weighted normal change penalty

(detail-preserving)

Hypothesis Want the normals to retain their spatial

direction with respect to the viewing direction and the light source

Strokes constrain the rotation of normals

Find min. α s.t. ρ(n(α)) = Itrg

Stroke smoothness and thick strokes Weaken the link between stroke and the rest

of surface

w/2-w/2

h(r)

r

x0

C

P(x0)

(detail-preserving)

(1-c)/d = f

Detail-preserving

(detail-preserving)

Stroke constrain

xi

xj

Constrain the new tangent

xi

xj

Constraint the projected position of

P(p) = P(ap1 + (1-a) p2)

p

C

P(p)

p1 p2

P(p1)

P(p2)

Realization of Stroke Attributes Stroke smoothness and thick strokes Silhouette strokes Interaction with highlights Highlight motion strokes

Adaptive refinement

441 vrtx.

1302 vrtx.

Adaptive √3-subdivision

Result MacBook Pro + 2GHx Intel Core Duo processor Performance issues

Stroke size, ROI setting, mesh size, degree of adaptive refinement

Conclusions and Future Work Shading-based surface editing

A direct and intuitive UI to modify surface Intuitive shading strokes

Future work Blur stroke

END

END