Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019....

48
Scalable Virtual Ray Lights Rendering for Participating Media Nicolas Vibert Adrien Gruson Heine Stokholm Troels Mortensen Wojciech Jarosz Toshiya Hachisuka Derek Nowrouzezahrai McGill University Luxion VIA University College Dartmouth College The University of Tokyo

Transcript of Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019....

Page 1: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Scalable Virtual Ray Lights Rendering for Participating Media

Nicolas Vibert Adrien Gruson Heine Stokholm Troels Mortensen Wojciech Jarosz Toshiya Hachisuka Derek Nowrouzezahrai

McGill University Luxion VIA University College Dartmouth College The University of Tokyo

Page 2: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MOTIVATION: Surface and Volume interaction2

Page 3: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MOTIVATION: Volume interaction only3

Page 4: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

VOLUMETRIC RENDERING4

Sensor

Image Plane𝜔

𝐿𝑚(𝑥, 𝜔) = න0

𝑠

𝑇𝑟(𝑢)𝐿𝑖(𝑥𝑢,𝜔)𝑑𝑢

𝑥𝑢

s

Page 5: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

VOLUMETRIC RENDERING: Rendering techniques

• Path tracing / Bidirectional path tracing

5

Page 6: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

VOLUMETRIC RENDERING: Many techniques6

• Path tracing / Bidirectional path tracing

• Density estimation:• Volumetric Photon Mapping

• Photon Beam

• Photon Planes

• “Higher-order geometric primitives”

Page 7: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

VOLUMETRIC RENDERING: Many techniques7

• Path tracing / Bidirectional path tracing

• Density estimation:• Volumetric Photon Mapping

• Photon Beam

• Photon Planes

• “Higher-order geometric primitives”

• Many lights:• Virtual point lights

• Virtual spherical lights

• Virtual ray lights

• “Higher-order geometric primitives”

Page 8: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS

• Many-light techniques have been introduced in “instant radiosity” [Keller et al. 1997]

• Indirect illumination as a sum of direct illumination of virtual lights

8

𝝎

Page 9: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS9

Virtual point light contribution

𝐿𝑚𝑉𝑃𝐿 =

)𝑉𝑃𝐿(𝑦, x𝑢)|𝑦 − 𝑥𝑢|

2𝑝(x𝑢

𝑦

𝑥𝑢

Page 10: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

𝐿𝑚𝑉𝑃𝐿 =

)𝑉𝑃𝐿(𝑦, x𝑢)|𝑦 − 𝑥𝑢|

2𝑝(x𝑢¥

MANY LIGHTS10

∝1

𝑦 − 𝑥𝑢2

𝑥𝑢

𝑦

Virtual point light contribution

Page 11: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS

• VPL vs. short VRL

11

Page 12: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS: VRL

Virtual ray lights contribution

12

𝐿𝑚VRL = න

0

𝑠

න0

𝑡 )𝑉𝑅𝐿(𝑢, 𝑣

𝑤 𝑢, 𝑣 2d𝑣d𝑢

𝑢

𝑣𝑉𝑅𝐿

Page 13: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS: VRL

Virtual ray lights contribution

13

𝐿𝑚𝑉𝑅L =

)𝑉𝑅𝐿(𝑢, 𝑣

)𝑤 𝑢, 𝑣 2𝑝(𝑢, 𝑣

𝑝 𝑢, 𝑣 ∝ 𝑤 𝑢, 𝑣 −2

𝑢

𝑣𝑉𝑅𝐿

Page 14: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS: VRL vs. VPL14

VRL VPL

Equal rendering time

Page 15: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS15

Realistic rendering : • Unmanageable amount of virtual lights• Cost linear with lights

Page 16: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

MANY LIGHTS16

Aim: • Sub-linear cost• Scalable methods

[Walter et al. 2005][Walter et al. 2006][Walter et al. 2012][Hasan et al. 2007][Ou et al. 2011][Bus et al. 2015]

Realistic rendering:• Unmanageable amount of virtual lights• Cost linear with lights

Page 17: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

SCALABILITY17

Page 18: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

SCALABILITY18

INDIVIDUAL LIGHTS

Page 19: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

SCALABILITY19

CLUSTERS

INDIVIDUAL LIGHTS

Page 20: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

SCALABILITY20

CLUSTERS

INDIVIDUAL LIGHTS

Page 21: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

RELATED WORKS21

Previous works have already explored a combination of VRLs with scalable techniques:

• Adaptive light-slice for virtual ray light [Frederickx al. 2015]

• Adaptive matrix column sampling and completion for rendering participating media [Huo et al. 2016]

Page 22: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound22

)𝑉𝑅𝐿(𝑢,𝑣

)𝑤 𝑢,𝑣 2𝑝(𝑢,𝑣< B

B=?

Page 23: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound

Φ𝑓(𝑢, 𝑣): Constant within the VRL cluster

23

Φ𝑓(𝑢,𝑣)𝑇𝑟(𝑢)𝑇𝑟(𝑤 𝑢,𝑣 )

)𝑤 𝑢,𝑣 2𝑝(𝑢,𝑣< B

Page 24: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound

Φ𝑓(𝑢, 𝑣): Constant within the VRL cluster

𝑇𝑟(𝑢): Impossible to control, assuming worst case => 1

24

Φ𝑓(𝑢,𝑣)𝑇𝑟(𝑢)𝑇𝑟(𝑤 𝑢,𝑣 )

)𝑤 𝑢,𝑣 2𝑝(𝑢,𝑣< B

Page 25: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound

Φ𝑓(𝑢, 𝑣): Constant within the VRL cluster

𝑇𝑟(𝑢): Impossible to control, assuming worst case => 1

𝑇𝑟(𝑤 𝑢, 𝑣 ): Based on the min distance

=> ℎ𝑚𝑖𝑛 ≤ 𝑤𝑘(𝑢, 𝑣)

=> 𝑇𝑟(𝑤 (𝑢, 𝑣)) < 𝑇𝑟(ℎ𝑚𝑖𝑛)

25

AABB

ℎ𝑚𝑖𝑛

𝑣

𝑢

𝑤 (𝑢, 𝑣)

Φ𝑓(𝑢,𝑣)𝑇𝑟(𝑢)𝑇𝑟(𝑤 𝑢,𝑣 )

)𝑤 𝑢,𝑣 2𝑝(𝑢,𝑣< B

Page 26: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Φ𝑓(𝑢, 𝑣): Constant within the VRL cluster

𝑇𝑟(𝑢): Impossible to control, assuming worst case => 1

𝑇𝑟(𝑤 𝑢, 𝑣 ): Based on the min distance

=> ℎ𝑚𝑖𝑛 ≤ 𝑤𝑘(𝑢, 𝑣)

=> 𝑇𝑟(𝑤 (𝑢, 𝑣)) < 𝑇𝑟(ℎ𝑚𝑖𝑛)

Φ𝑓(𝑢,𝑣)𝑇𝑟(𝑢)𝑇𝑟(𝑤 𝑢,𝑣 )

)𝑤 𝑢,𝑣 2𝑝(𝑢,𝑣< B

Our solution: Upper bound26

AABB

ℎ𝑚𝑖𝑛

𝑣

𝑢

𝑤 (𝑢, 𝑣)

Page 27: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound27

𝑤𝑘 𝑢, 𝑣 2𝑝 𝑢, 𝑣=𝑤𝑘 𝑢, 𝑣 2𝑝(𝑢|𝑣)𝑝(𝑣)

Page 28: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound

Worst case when VRL and sensor ray are parallel.

)𝑝(𝑣 <1

𝐿𝑚𝑎𝑥, with 𝐿𝑚𝑎𝑥 the maximum length inside the cluster.

28

𝑤𝑘 𝑢, 𝑣 2𝑝 𝑢, 𝑣=𝑤𝑘 𝑢, 𝑣 2𝑝(𝑢|𝑣)𝑝(𝑣)

Page 29: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound

Worst case when VRL and sensor ray are parallel.

)𝑝(𝑣 <1

𝐿𝑚𝑎𝑥, with 𝐿𝑚𝑎𝑥 the maximum length inside the cluster.

29

𝑤𝑘 𝑢, 𝑣 2𝑝 𝑢, 𝑣=𝑤𝑘 𝑢, 𝑣 2𝑝(𝑢|𝑣)𝑝(𝑣)

Page 30: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound30

AABB𝑣

𝑤 (𝑢, 𝑣)

Θ

𝑢

തℎ

ത𝑢

𝑠

Page 31: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound31

AABB𝑣

𝑤 (𝑢, 𝑣)

Θ

𝑢

തℎ

ത𝑢

𝑠

𝑤 𝑢, 𝑣 2𝑝 𝑢 𝑣 = ഥℎ(ഥℎ2+ഥ𝑢2)

Θ(ഥℎ2+ഥ𝑢2)

Page 32: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound32

AABB𝑣

𝑤 (𝑢, 𝑣)

Θ

𝑢

തℎ

ത𝑢

𝑤 𝑢, 𝑣 2𝑝 𝑢 𝑣 = ഥℎ(ഥℎ2+ഥ𝑢2)

Θ(ഥℎ2+ഥ𝑢2)

= ഥℎ

Θ

𝑠

Page 33: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound33

AABB

ℎ𝑚𝑖𝑛

𝑤 𝑢, 𝑣 2𝑝 𝑢 𝑣 = ഥℎ(ഥℎ2+ഥ𝑢2)

Θ(ഥℎ2+ഥ𝑢2)

= ഥℎ

Θതℎ > ℎ𝑚𝑖𝑛

𝑠

Θ

Page 34: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound34

AABB

ℎ𝑚𝑖𝑛

𝑤 𝑢, 𝑣 2𝑝 𝑢 𝑣 = ഥℎ(ഥℎ2+ഥ𝑢2)

Θ(ഥℎ2+ഥ𝑢2)

= ഥℎ

Θതℎ > ℎ𝑚𝑖𝑛

𝑠

Θ

Page 35: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound35

AABB

ℎ𝑚𝑖𝑛

𝑤 𝑢, 𝑣 2𝑝 𝑢 𝑣 = ഥℎ(ഥℎ2+ഥ𝑢2)

Θ(ഥℎ2+ഥ𝑢2)

= ഥℎ

Θതℎ > ℎ𝑚𝑖𝑛

𝑠

Θ𝑚𝑎𝑥

Page 36: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound36

AABB

ℎ𝑚𝑖𝑛

𝑤 𝑢, 𝑣 2𝑝 𝑢 𝑣 = ഥℎ(ഥℎ2+ഥ𝑢2)

Θ(ഥℎ2+ഥ𝑢2)

= ഥℎ

Θതℎ > ℎ𝑚𝑖𝑛Θ < Θ𝑚𝑎𝑥

𝑠

Θ𝑚𝑎𝑥

Page 37: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Upper bound37

B = Φ𝑓 𝑢,𝑣 𝑇𝑟 hmin ΘmaxLmax

ℎ𝑚𝑖𝑛

Page 38: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Light tree38

Agglomerative approach [Walter et al. 2008]:

- Not multithreaded

- Does not scale with high node overlapping 𝑂(𝑁2)

What we need:

- Fast/Parallelizable

- Agglomerative principal

Page 39: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Light tree39

Page 40: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Light tree

Step 1: Partition the space with sorting

40

Page 41: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Light tree41

Step 2: Build local light tree that minimize the metric

Page 42: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Our solution: Light tree42

Step 3: Build final tree with agglomerative process

Page 43: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Results

• Equal time comparison• VPL with LC

• VRL

• Two metrics• RMSE: sensitive to fireflies

• SMAPE: robust to fireflies

• Isotropic medium, only medium-medium interactions

43

Page 44: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Results44

Reference VPL LC (1M) 344 secs

RMSE: 9.01SMAPE: 3.25

VRL LC (100k)370 secs

RMSE: 2.70SMAPE: 4.31

VRL (10k)323 secs

RMSE: 5.46SMAPE: 9.93

Page 45: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Results45

VPL LC (1M) - 147 secsRMSE: 0.33

SMAPE: 2.44

VRL LC (100k) - 121 secsRMSE: 0.01SMAPE 1.88

VRL (10k) - 147 secsRMSE: 0.05SMAPE 9.11

0.0

> 0.05

SMAPE

Page 46: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Results46

Re

lati

ve s

pe

ed

-up

STARCAISE VEACH-LAMP VEACH-LAMP BATHROOM

VRL count VPL count

Page 47: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Summary

Contributions:

• New bound for VRL cluster• Efficient tree construction• X10 Speedup

47

Page 48: Scalable Virtual Ray Lights Rendering for Participating Mediathachisu/vrlcuts_slides.pdf · 2019. 7. 22. · Scalable Virtual Ray Lights Rendering for Participating Media Nicolas

Questions ?

48