Transcript of Inverse Volume Rendering with Material Dictionaries Ioannis Gkioulekas 1 Shuang Zhao 2 Kavita Bala 2...
- Slide 1
- Inverse Volume Rendering with Material Dictionaries Ioannis
Gkioulekas 1 Shuang Zhao 2 Kavita Bala 2 Todd Zickler 1 Anat Levin
3 1 Harvard 3 Weizmann 2 Cornell 1
- Slide 2
- Most materials are translucent 2 jewelry skin architecture
Photo credit: Bei Xiao, Ted Adelson food
- Slide 3
- We know how to render them 3 Monte-Carlo rendering material
parameters Veach 1997, Dutr et al. 2006 ? rendered image
- Slide 4
- We show how to measure them 4 inverse rendering material
parameters rendered image captured photograph
- Slide 5
- Our contributions 5 material 1. exact inverse volume rendering
with arbitrary phase functions! 2. validation with calibration
materials known parameters 3. database of broad range of materials
thinthick non- dilutable solids
- Slide 6
- material sample Why is inverse rendering so hard? 6 radiative
transfer random walk of photons inside volume volume light
transport has very complex dependence material parameters thinthick
non- dilutable solids
- Slide 7
- thinthick non- dilutable solids Light transport approximations
7 Previous approach: single-scattering random walk of photons
inside volume single-bounce random walk Narasimhan et al. 2006
- Slide 8
- Light transport approximations 8 Previous approach: diffusion
Jensen et al. 2001 Papas et al. 2013 isotropic distribution of
photons parameter ambiguity material 1 material 2 random walk of
photons inside volume thinthick non- dilutable solids
- Slide 9
- Inverse rendering without approximations 9 random walk of
photons inside volume exact inversion of random walk thinthick non-
dilutable solids
- Slide 10
- Our approach 10 appearance matching ii. operator-theoretic
analysis i. material representation iii. stochastic
optimization
- Slide 11
- Background 11 phase function p() scattering coefficient s
extinction coefficient t m = ( t s p()) random walk of photons
inside medium
- Slide 12
- Papas et al. 2013 Phase function parameterization 12 not
general enough Henyey-Greenstein lobes Chen et al. 2006 Donner et
al. 2008 Fuchs et al. 2007 Goesele et al. 2004 Gu et al. 2008
Hawkins et al. 2005 Holroyd et al. 2011 McCormick et al. 1981 Pine
et al. 1990 Prahl et al. 1993 Wang et al. 2008 Gkioulekas et al.
2013 Narasimhan et al. 2006 Jensen et al. 2001 Previous approach:
single-parameter families
- Slide 13
- m = q q m q p = q q p q D = {m 1, m 2, , m Q } Dictionary
parameterization 13 tent phase functions D = {p 1, p 2, , p Q }
p1p1 p2p2 p3p3 p4p4 p5p5 p6p6 p7p7 p8p8 p9p9 p 10 p 11 dictionary
of arbitrary p similarly for t and s 11 22 33 44 55 66 77 88 99 10
11 D phase functions materials t = q q t,q s = q q s,q
- Slide 14
- Our approach 14 appearance matching ii. operator-theoretic
analysis i. material representation iii. stochastic optimization m
= q q m q
- Slide 15
- Operator-theoretic analysis 15 m = ( t s p()) random walk of
photons inside medium discretized random walk paths propagation
step
- Slide 16
- total radiance K() = q q K q Operator-theoretic analysis 16 m =
( t s p()) discretized random walk paths propagation step L(x, )
radiance at all medium points and directions L n+1 (x, ) = L n (x,
)K rendering operator R = (I - K) -1 L input L = n L n L(x, ) = R L
input (x, ) radiance after n steps radiance after n+1 steps R()= (I
- q q K q ) -1 dictionary representation: m = q q m q
- Slide 17
- Our approach 17 appearance matching ii. operator-theoretic
analysis i. material representation iii. stochastic optimization m
= q q m q R()= (I - q q K q ) -1
- Slide 18
- Stochastic optimization 18 appearance matching analytic
operator expression for gradient! R() render()single-step q
render() R()KqKq gradient descent optimization for inverse
rendering min photo - render() 2
- Slide 19
- Stochastic optimization 19 exact gradient descent for k = 1, ,
N, k = k - 1 - a k end N = a few hundreds several CPU hours * =
intractable exact
- Slide 20
- Stochastic optimization 20 Monte-Carlo rendering to compute 10
2 samples noisy + fast 10 4 samples 10 6 samples accurate +
slow
- Slide 21
- Stochastic optimization 21 exact gradient descent for k = 1, ,
N, k = k - 1 - a k end N = a few hundreds several CPU hours * =
intractable stochastic gradient descent for k = 1, , N, k = k - 1 -
a k end N = a few hundreds few CPU seconds * = solvable
exactnoisy
- Slide 22
- Theory wrap-up 22 appearance matching ii. operator-theoretic
analysis i. material representation iii. stochastic optimization m
= q q m q R()= (I - q q K q ) -1 noisy min photo - render() 2
- Slide 23
- Our contributions 23 material 1. exact inverse volume rendering
with arbitrary phase functions! 2. validation with calibration
materials known parameters 3. database of broad range of materials
thinthick non- dilutable solids
- Slide 24
- Measurements 24 multiple lighting multiple viewpoints
appearance matching min photo - render() 2
- Slide 25
- Acquisition setup 25 material sample frontlighting backlighting
camera
- Slide 26
- Acquisition setup 26 bottom rotation stage top rotation stage
material sample frontlighting backlighting material sample
frontlighting camera backlighting bottom rotation stage top
rotation stage camera
- Slide 27
- Validation 27 Frisvad et al. 2007 polystyrene monodispersions
aluminum oxide polydispersions very precise dispersions (NIST
Traceable Standards) calibration materials known parameters Mie
theory size % particle material medium material
- Slide 28
- Parameter accuracy 28 polystyrene 1polystyrene 2polystyrene
3aluminum oxide all parameters estimated within 4% error comparison
of ground-truth and measured parameters ground-truth measured
Henyey-Greenstein fit -0 p()
- Slide 29
- Matching novel measurements 29 captured rendered rendered with
HGprofiles polystyrene 3 comparison of captured and rendered images
images under unseen geometries predicted within 5% RMS error
ground-truth measured Henyey-Greenstein fit
- Slide 30
- Our contributions 30 material 1. exact inverse volume rendering
with arbitrary phase functions! 2. validation with calibration
materials known parameters 3. database of broad range of materials
thinthick non- dilutable solids
- Slide 31
- thinthick non- dilutable solids Measured materials 31 mustard
whole milk shampoo hand cream coffee wine robitussin olive oil
curacao mixed soap milk soap liquid clay reduced milk
- Slide 32
- Measured phase functions 32 whole milkreduced milk mustard
shampoohand cream liquid claymilk soapmixed soapglycerine soap
robitussin coffee olive oil curacao wine -0 p() measured
Henyey-Greenstein fit
- Slide 33
- whole milkreduced milk mustard shampoohand cream liquid
claymilk soapmixed soapglycerine soap robitussin coffee olive oil
curacao wine Measured phase functions 33 -0 p() measured
Henyey-Greenstein fit
- Slide 34
- Synthetic images 34 mixed soap glycerine soapolive
oilcuracaowhole milk rendered image
- Slide 35
- Synthetic images 35 chromaticity
- Slide 36
- Synthetic images 36 mixed soap glycerine soapolive
oilcuracaowhole milk rendered image
- Slide 37
- Effect of phase function 37 mixed soap measured phase function
Henyey-Greenstein fit -0 p() rendered image chromaticity measured
Henyey-Greenstein fit
- Slide 38
- Discussion 38 faster capture and convergence: trade-offs
between accuracy, generality, mobility, and usability more
interesting materials: more general solids, heterogeneous volumes,
fluorescing materials other setups: alternative lighting (basis,
adaptive, high- frequency), geometries, or imaging (transient
imaging)
- Slide 39
- Take-home messages 39 material 1. exact inverse volume
rendering with arbitrary phase functions! 2. validation with
calibration materials known parameters 3. database of broad range
of materials thinthick non- dilutable solids
- Slide 40
- Acknowledgements 40 Henry Sarkas (Nanophase) Wenzel Jakob
(Mitsuba) Funding: National Science Foundation European Research
Council Binational Science Foundation Feinberg Foundation Intel
Amazon http://tinyurl.com/sa2013-inverse Database of measured
materials:
- Slide 41
- Error surface 41 appearance matching min photo - render()
2
- Slide 42
- Light generation 42 MEMS light switch RGB combiner blue (480
nm) laser green (535 nm) laser red (635 nm) laser