PHOTOREALISTIC ANIMATION RENDERING WITH POPULATION MONTE CARLO
156
PHOTOREALISTIC ANIMATION RENDERING WITH POPULATION MONTE CARLO ENERGY REDISTRIBUTION by Yu-Chi Lai A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Sciences) at the UNIVERSITY OF WISCONSIN–MADISON 2009
Transcript of PHOTOREALISTIC ANIMATION RENDERING WITH POPULATION MONTE CARLO
PHOTOREALISTIC ANIMATION RENDERING WITH POPULATION MONTE
CARLO ENERGY REDISTRIBUTION
by
Yu-Chi Lai
A dissertation submitted in partial fulfillment of
the requirements for the degree of
Doctor of Philosophy
(Computer Sciences)
at the
UNIVERSITY OF WISCONSIN–MADISON
2009
c© Copyright by Yu-Chi Lai 2009
All Rights Reserved
i
To my dad
ii
ACKNOWLEDGMENTS
I thanks to my adviser Charles Dyer for his support on my PhD work. Without his support
and guidance, I probably gave up this work when Stephen Chenney left UW-Madison. I started
to work with Chuck about 6 years ago when I took computer vision class with him. From that
moment, Chuck has been my best mentor and has always been there when I needed advice. Chuck
has always provided me courage and proper directions to facethe difficulties when pursuing the
PhD. degrees in ECE and CS. He has taught me so much over the years from doing research to
writing papers. I really appreciate everything he has done for me.
I thanks to my original adviser Stephen Chenney for his inspiration and valuable guidance
when I started my research in computer graphics. I feel so fortunate to have had the opportunity
to work with Stephen because he is always very supportive, patient and encouraging. Stephen
taught me a lot not only in computer graphics, but also in manyaspects in life. He is very sharp
as a researcher and extremely nice as a person. His stimulating insights and discussions helped
the research tremendously. His kindness and encouragementmade the experience of working with
him very enjoyable.
I would like to thank my other committee members, Mike Gleicher, Yu-Hen Hu, Li Zhang, and
Jerry Zhu. Mike is a strong believer of lab environment and makes the lab a pleasant place to work,
from which I have benefited so much. Thank Yu-Hen, Li, and Jerry for serving on my committee
and taking the time to review my thesis.
I can not express sufficiently for how much I have enjoyed our computer graphics/vision group
environment and how much I have learned from all our group members. I would like to mention
Michael Wallick, Rachel Heck, Lucas Kovar, Mankyu Sung, ShaoHua Fan, and Feng Liu for the
discussions and sharing of their experience on prelim and oral exams. I also appreciate ShaoHua
iii
Fan and Feng Liu for the “lunch meetings” and all the fun, relaxing chats we had. Special thank
goes to ShaoHua for giving the inspiration and basic foundation for pursuing this topics.
I also thanks for Daniel Wong, Theophilus Benson, and Brandon Smith for their patience to
review my papers and dissertation. This helped me remove thefrustrated grammatical mistakes
commonly made in my writing.
Finally, my deepest gratitude goes to my family, for everything.
DISCARD THIS PAGE
iv
TABLE OF CONTENTS
Page
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
NOMENCLATURE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 The Global Illumination Problem . . . . . . . . . . . . . . . . . . . .. . . . . . . 21.2 Summary of Contributions . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 41.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 5
2 Monte Carlo Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1 Monte Carlo Methods: A Brief History . . . . . . . . . . . . . . . . .. . . . . . . 72.2 Monte Carlo Integration . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 8
2.2.1 Estimators and Their Properties . . . . . . . . . . . . . . . . . .. . . . . 102.3 MCMC and Metropolis-Hastings Sampling . . . . . . . . . . . . . .. . . . . . . 112.4 D-Kernel Population Monte Carlo . . . . . . . . . . . . . . . . . . . .. . . . . . 13
3 Basics for Global Illumination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1 Radiometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163.2 Bidirectional Scattering Distribution Function . . . . .. . . . . . . . . . . . . . . 173.3 The Rendering Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 193.4 Monte Carlo Integrations for Global Illumination . . . . .. . . . . . . . . . . . . 22
3.4.1 Path Integral Formulation of the Rendering Equation .. . . . . . . . . . . 223.4.2 Monte Carlo Integrations . . . . . . . . . . . . . . . . . . . . . . . .. . . 263.4.3 Raytracing-Based Monte Carlo Rendering Algorithms .. . . . . . . . . . 28
3.5 Markov Chain Monte Carlo Algorithms for Global Illumination . . . . . . . . . . 31
v
Page
3.5.1 Formulation for Markov Chain Monte Carlo Algorithms .. . . . . . . . . 313.5.2 Markov Chain Monte Carlo Global Illumination Methods. . . . . . . . . 35
3.6 Population Monte Carlo Rendering . . . . . . . . . . . . . . . . . . .. . . . . . . 373.7 Global Illumination Animation Rendering . . . . . . . . . . . .. . . . . . . . . . 38
3.7.1 Hybrid Methods of Image-Based Rendering and Global Illumination . . . . 393.7.2 Radiosity Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .403.7.3 Virtual Light Sources . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 413.7.4 Photon Shooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.7.5 Irradiance Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .443.7.6 Parallel Computation and Accelerate Ray Tracing . . . .. . . . . . . . . . 453.7.7 Reuse Path Information . . . . . . . . . . . . . . . . . . . . . . . . . .. . 473.7.8 Decouple the Rendering and Displaying Processes . . . .. . . . . . . . . 493.7.9 Hardware Rendering . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50
4 Photorealistic Image Rendering withPopulation Monte Carlo EnergyRedistribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.1 Population Monte Carlo Energy Redistribution . . . . . . . .. . . . . . . . . . . 534.1.1 PMC-ER Equal Deposition Algorithm . . . . . . . . . . . . . . . .. . . . 534.1.2 Energy Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .544.1.3 The Kernel Function for Each Path . . . . . . . . . . . . . . . . . .. . . . 564.1.4 Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .624.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 66
5 Efficient Schemes for Monte Carlo Markov Chain Algorithms in Global Illumination 68
5.1 New Schemes to MCMCs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .705.1.1 Population Monte Carlo Energy Redistribution with Equal Deposition . . . 705.1.2 New Lens Perturbation . . . . . . . . . . . . . . . . . . . . . . . . . . .. 715.1.3 Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .805.2.1 Comparison between Old Mutations and New Mutations . .. . . . . . . . 815.2.2 Comparison between Stratified Regeneration and All Three Regenerations . 835.2.3 Combine Everything into PMC-ER-E . . . . . . . . . . . . . . . . .. . . 83
5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 88
vi
Page
6 Physically-based Animation Rendering with Markov Chain Monte Carlo . . . . . . 90
6.1 Animation Formulation for Markov Chain Monte Carlo . . . .. . . . . . . . . . . 926.2 Population Monte Carlo Energy Redistribution . . . . . . . .. . . . . . . . . . . 94
6.2.1 Population Monte Carlo Energy Redistribution with Equal Deposition . . . 946.2.2 Time Perturbation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .956.2.3 Iteratively Distribute Samples in a Frame-by-Frame Manner . . . . . . . . 99
6.3 Animation Rendering System . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 1006.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.3.2 Host Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.3.3 Client Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1016.3.4 Acceleration Temporal and Spatial Structures in Time. . . . . . . . . . . 1026.3.5 Sample Recorder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1036.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 107
7 Discussion and Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
7.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 1157.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 117
LIST OF REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
DISCARD THIS PAGE
vii
LIST OF TABLES
Table Page
4.1 Statistical information when rendering images with PMC-ER-E and ERPT . . . . . . 66
5.1 Statistical information when rendering images with oldmutations and new mutations. 81
5.2 Statistical information when rendering images with stratified regeneration and withthree regenerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 82
5.3 Measurements comparing PMC-ER with PMC-ER-E using all the new schemes. . . . 82
6.1 Scene specific paramesters. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 105
6.2 Statistical information when rendering animation withtime mutation and without timemutation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
DISCARD THIS PAGE
viii
LIST OF FIGURES
Figure Page
2.1 The Metropolis sampling algorithm . . . . . . . . . . . . . . . . . .. . . . . . . . . 12
2.2 The generic D-Kernel Population Monte Carlo algorithm.. . . . . . . . . . . . . . . 13
3.1 The input and output geometrical relation used to describe bidirectional scatteringdistribution function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 18
3.2 Image sweep diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 20
3.3 The geometric relation fro three-point light transportequation. . . . . . . . . . . . . . 23
4.1 Population Monte Carlo Energy Redistribution Equal Deposition iteration loop. . . . . 55
4.2 An example of using lens perturbation to reconstruct a path. . . . . . . . . . . . . . . 57
4.3 An example of using caustics perturbation to reconstruct a caustics path. . . . . . . . . 58
4.4 Cornell Box images rendered with PMC-ER-E and ERPT. . . . .. . . . . . . . . . . 62
4.5 Dragon images rendered with PMC-ER-E and ERPT. . . . . . . . .. . . . . . . . . . 63
4.6 Room images rendered with PMC-ER-E and ERPT. . . . . . . . . . .. . . . . . . . 64
5.1 Modified Population Monte Carlo Energy Redistribution Equal Deposition iterationloop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
5.2 An example of using the new lens perturbation to reconstruct a caustics path . . . . . 73
5.3 An example of constructing a caustics path from a light source. . . . . . . . . . . . . 77
5.4 Cornell Box images rendered with new-scheme PMC-ER-E and original PMC-ER-E. . 84
5.5 Dragon images rendered with new-scheme PMC-ER-E and original PMC-ER-E . . . . 85
ix
Figure Page
5.6 Room images rendered with new-scheme PMC-ER-E and original PMC-ER-E. . . . . 86
6.1 The PMC-ER Equal Deposition iteration loop for a client process when rendering ananimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.2 An example of using time perturbation to reconstruct a path. . . . . . . . . . . . . . . 110
6.3 Condor animation rendering flow chart. . . . . . . . . . . . . . . .. . . . . . . . . . 111
6.4 The keyframed rigit transformation to represent the animation of object. . . . . . . . . 112
6.5 The diagram to represent a movie-like sample recorder. .. . . . . . . . . . . . . . . . 112
6.6 A sequence of Cornell Box images rendered with time mutation and without timeperturbation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 113
6.7 A Cornell Box images rendered with motion blur. . . . . . . . .. . . . . . . . . . . . 113
6.8 The 60-th frame from the animations rendered the CB2, room, chess board, and base-ment scenes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
DISCARD THIS PAGE
x
NOMENCLATURE
Q Total energy of photon events
Φ Radiant power or flux
E(·)/E(X)Irradiance or energy of a light transport path when being constructed
I Intensity on a pixel or the integral of a function
L/L(X) Radiance or contribution of a light transport path
fr Bidirectional Reflectance Distribution Function (BRDF)
fs Bidirectional Scattering Distribution Function (BSDF)
ft Bidirectional Transmission Distribution Function (BTDF)
xy A 3D surface point
XY A 3D surface sample point
x y A light transport path
X Y A sample light transport path
S The set of 3D finite surfaces
M The set of valid 3D object surface in a scene
Rn N-dimensional real space
xi
Ω the domain of integration
N(xt) Surface normal at the surface point ofxt
dA(xt) The differential imaginary area around the surface pointxt
dA⊥(xt) The differential imaginary area projected onto the normal direction around the surface
pointxt
ω 3D direction
σ(ω) The solid angle in the direction ofω
Tr(xt, ω) Transport function to find the first intersection surface point from current surface
pointxt in the direction ofω.
G(x,x′) The geometry term between two surface points
V (x,x′) The visibility term between two surface points
p(·) The density Distribution function of the sample
E [·] Expectation of a function
V[·] Variance of a function
σ[·] Standard deviation of a function
T [·] The timing for doing the job
Eff [·] The efficiency of doing the job
P − Eff [·] Perceptually-based mean squared efficiency of doing the job
MSE [·] The mean square error
π(f)∫
Ω f(x)π(x)dx the integral of a function based on the distribution function π(x)
xii
β(·) Bias
IN Estimator forI with N samples
T (·|·) Tentative transition function
Tr(·|·) Transition function
K(·|·) Kernel function
A(·|·) Acceptance probability function
E Average path energy in a rendered result
ed Deposit energy for each mutation
EIP Average path energy for deposit on the image plane
pIP Probability to choose a pixel position from the image plane
α The mixture weight for choosing a perturbation method.
dh The perturbation radius in spatial domain
th The perturbation radius in temporal domain
γ The mixture weight for pixel-position distribution.
err Error rate.
W The sensor response function
µ Area product measurement
PT Path tracing
BPT Bidirectional path tracing
PM Photon mapping
xiii
IC Irradiance caching
PMC Population Monte Carlo
PMC-IP The Population Monte Carlo Image Plane sampling algorithm
PMC-HI The Population Monte Carlo Hemispherical Integral sampling algorithm
PMC-PT Population Monte Carlo Path Tracing
MCMC Markov Chain Monte Carlo
MLT Metropolis Light Transport
ERPT Energy Redistribution Path Tracing
PMC-ER Population Monte Carlo Energy Redistribution
PMC-ER-EPopulation Monte Carlo Energy Redistribution Equal Deposition
PMC-ER-BPopulation Monte Carlo Energy Redistribution Balance Transfer
PDF Probability Distribution Function
SPP Samples Per Pixel
PHOTOREALISTIC ANIMATION RENDERING WITH POPULATION MONTE
CARLO ENERGY REDISTRIBUTION
Yu-Chi Lai
Under the supervision of Assistant Professor Li Zhang and Professor Charles R. Dyer
At the University of Wisconsin-Madison
Requirement of realism becomes higher and higher with the advance of computer hardwares and
algorithms. To generate photorealistic animation requires the estimation of a large number of
temperally and spatially correlated path integrals. In this dissertation we present a space-time
algorithm to exploit the spatial and temporal coherence among all path integrals based on the
Markov Chain Monte Carlo (MCMC) framework when rendering a physically-correct animation.
Although sample resue can improve the rendering efficiency,the choice of good mutation strategies
for MCMC algorithms is non-trivial and has a major impact on image quality. Thus, we adapt the
Population Monte Carlo framework into the energy redistribution algorithm to reuse information
from important samples with a mutation process whose mutation strategy is adapted on-the-fly.
The purposed algorithm is called Population Monte Carlo Energy Redistribution (PMC-ER) and
is self-tuning to a large extent. Through the progress of thedissertation, we also propose a new
lens perturbation to simplify the computation and control of caustics perturbation and possibly
increases the perturbation success rate and two path-regeneration methods aiming to concentrate
more computation on “high perceptual variance” regions and“hard-to-find-but-important” paths.
Additionally, a new perturbation technique calledtime perturbationis developed to explore the
temporal coherence among paths. Furthermore, in order to make animation rendering plausible,
we develop a parallel rendering system to distribute the iterative computational tasks to a pool of
computers. Each task is rendered with a set of task-based parameters without introducing bias. The
resulting animations are perceptually better than those rendered in a frame-by-frame manner. In
this dissertation we demonstrate that Population Monte Carlo Energy Redistribution can enhance
the rendering efficiency by automatically adjusting the rendering parameters when exploiting the
xiv
temporal and spatial coherence among light transport pathsand by concentrating more computation
on visually important regions on the image plane and in the path space.
Li Zhang Charles R. Dyer
xv
ABSTRACT
A physically-correct animation can be generated by estimating a large number of highly corre-
lated path integrals falling on the image plane at a sequenceof different periods of time. Therefore,
exploring temporal and spatial coherence among path integrals is very important for enhancing ren-
dering efficiency. Removing temporal artifacts is even moreimportant than spatial artifacts because
our visual perception is very sensitive to temporal artifacts. Thus, exploring temporal coherence
among path integrals is important to remove temporally perception-sensitive artifacts in anima-
tion rendering. In this dissertation we purpose a novel spac-time global illumination algorithm,
Population Monte Carlo Energy Redistribution which concentrates computation on important light
transport paths and automatically adjusts energy distributed extent for each light transport path.
We adapt statistical framework of Population Monte Carlo into the energy redistribution algorithm
to improve rendering efficiency. In the Population Monte Carlo framework, information collected
in previous iterations is used to guide subsequent iterations by adapting the kernel function to
approximate the target distribution without introducing bias into the final result. Based on this
framework, our algorithm automatically adapts the amount of energy redistribution at different
pixels of different frame and the extent over which energy isredistributed.
After applying MCMC algorithms to rendering static images,we observe that MCMC algo-
rithms are limited in achieving higher rendering efficiencydue to possibly high failure rate in
the caustics perturbation method and the stratified exploration of the image plane. We improve
MCMCs significantly by introducing a new lens perturbation and new path-regeneration methods.
The new lens perturbation method simplifies the computationand control of caustics perturba-
tion and possibly increases the perturbation success rate.The new path-regeneration methods
xvi
aim to concentrate more computation on “high perceptual variance” regions and “hard-to-find-but-
important” paths. We implement these schemes in the PMC-ER framework to demonstrate the
effectiveness of these improvements. Our results show thatrendering efficiency is improved with
these new schemes.
Since our perception is very sensitive to temporal artifacts, exploring temporal coherence
among light transport paths is very important to remove temporally perception-sensitive artifacts
in animation rendering. Using the contribution of a light path to all frames in an animation as the
sampling distribution function allows us to adapt Markov Chain Monte Carlo (MCMC) algorithms
to exploit the temporal and spatial coherences to generate aperceptually pleasant animation. As
a result, a new perturbation technique calledtime perturbationis developed to explore the tem-
poral coherence among paths. Furthermore, in order to make animation rendering plausible, we
distribute the iterative computational tasks to a pool of computers for parallel computation. Each
task is rendered with a set of parameters adapted according to the local properties of each task. We
demostrate that this local adatpation does not introduce bias statistically. The resulting animations
are perceptually better than those rendered in a frame-by-frame manner.
1
Chapter 1
Introduction
Currently, a large number of movies employ visual effects using computer graphics. Photo-
realistic rendering is critical for such visual effects. Bad rendering will make the audience lose
the sense of realism. In addition, product designers, architects, and illumination designers need
photorealistic rendering to predict the appearance of their products and designs. For example, an
architect needs to know the appearance of the interior of a room and the appearance of a building
under different weather conditions; a stage designer needsto understand the effects of different
lights on a concert stage; a car designer would like to see theappearance of a car in different il-
luminations. The main advantage of physically-based rendering is that the images generated can
accurately predict the appearance in the real world withoutactually setting up the real scenes. Fur-
thermore, due to advances in computer hardware the demand ofrealism for real-time applications
such as computer games and virtual reality walk-throughs becomes higher and higher. Physically-
based rendering can provide the base for developers to understand the fundamental principles of
image rendering in order to enhance realism and visual appeal. And some real-time techniques
such as light maps and environment maps need photorealisticrendering to generate proper media
in the preprocessing phase. Efficiently rendering high-quality images is a long standing goal in
global illumination. Therefore, our goal is to create an unbaised rendering algorithm which en-
hances the rendering efficiency in generating a physically correct animation. The reminder of this
chapter presents the statement of problems, a brief summaryof contributions, and an overview of
the subsequent chapters.
2
1.1 The Global Illumination Problem
The goal of our algorithm is to efficiently render a photorealistic animation through off-line
computation. Thus, the input to our problem is a complete scene with animated objects. The output
is a rendered photorealistic animation. Since the computation is off-line, it is proper to assume
that all information related to the animated scenes is knowna priori. The animated information
includes change in the intensity of a light, change in the material properties of an object, movement
of lights and objects, and movement of the viewpoint. With this assumption, we can fully utilize all
information in the scene, especially the information contained in frames preceding and following
the current frame. In order to generate physically accurateanimations, we need to simulate all
inter-reflections between the lights and objects in the scene model in different periods of time; this
is called the global illumination problem [8, 142].
The rendering equation [77] is the foundation for physically-based rendering in global illumina-
tion. There are two different types of methods: finite element and Monte Carlo methods. Finite el-
ement methods, or radiosity algorithms [53], are most effective in scenes containing purely diffuse
surfaces. However, it is difficult to generalize finite element algorithms for non-diffuse surfaces
and complex geometries both practically and theoretically. On the other hand Monte Carlo meth-
ods can handle general light-surface interactions and different geometric representations. They
are ideal for solving global illumination problem for a complex scene. Kajiya [77] proposedpath
tracing, which is the first unbiased Monte Carlo algorithm. Over the years, new Monte Carlo
algorithms with various trade-offs in speed, accuracy, andgenerality have been developed.
Past results demonstrate that Monte Carlo is the most general and robust method. However, the
results also demonstrate that if too few samples are taken when using Monte Carlo algorithms, vis-
ible noise appears in the resulting images and the noise level is reduced only slowly as the number
of samples increases. The results shown in [143, 24] demonstrate that to render a single complex
scene takes many hours. Furthermore, professionals renderan animation in a frame-by-frame man-
ner [7, 140]. This means that all samples are generated independently and the construction of the
samples have to be repeated and started from scratch for eachframe. As a result, Monte Carlo is
3
rarely exploited for the visualization of dynamic scenes inwhich photorealism is important. Since
the samples are highly correlated in both the spatial and temporal domains, it is obvious that not
using spatial or temporal information among samples is a waste – the information on high contri-
bution samples which can help generate good samples for the neighborhood of a pixel in current
frame or in consecutive frames is not being exploited. More seriously, not exploiting the tempo-
ral coherence among samples and frames causes temporal artifacts (flickering and shimmering).
Our visual perception has evolved to be sensitive to these kinds of artifacts. Our results show that
exploiting the temporal and spatial coherence among samples enhances the speed of computation
and reduces noise in rendered results.
The global illumination community has started to pay more attention to the temporal coherence
among samples to accelerate the rendering process. Some have started to study simple animation
cases where only the light sources are moving or only the viewpoint is changing [12, 125, 123, 124,
62, 95]. Samples are reused by reevaluating the contribution of each sample or reweighting each
sample based on the multiple-importance framework. The limitation in which objects are allowed
to move limits the usefulness of the algorithms. Some attempts have been made to reuse samples
by copying their contribution across several frames [105, 144]. Although those algorithms speed
up the rendering process by reducing the number of rays beingtraced and reducing the flickering
artifacts, they introduce bias (errors) into the renderingresults since the copied samples may not
be valid or reweighted properly across frames.
To the best of our knowledge, there is no unbiased global illumination algorithm proposed to
take advantage of temporal coherence for rendering a complete and complex animated scene by
considering all possible light transport paths through theentire animation.
In this disseration we present a novel and unbiased rendering algorithm, Population Mon-
teCarlo Energy Redistribution (PMC-ER), based on statistical Population Monte Carlo (PMC)
methods to take advantage of temporal and spatial coherencefor rendering a complete and com-
plex animated scene by considering all possible light transport paths through the entire animation.
PMC-ER techniques offer three major advantages over existing methods: 1) they reduce spatial
4
and temporal noise by choosing samples correlatedly acrossframes and pixels without introduc-
ing bias; 2) they provide a natural way to continue exploringimportant light transport paths and
remove paths with low contribution; and 3) they can automatically adjust the rendering param-
eters used to explore the spatial and temporal coherence. All these can enhance the rendering
efficiency in physically-based animation rendering. We demonstrate that our rendered animations
are perceptually pleasant.
1.2 Summary of Contributions
We adapt the population Monte Carlo framework into energy redistribution to render a physi-
cally correct animation. Our main contributions are the following:
• Population Monte Carlo Energy Redistribution: Population Monte Carlo Energy Redis-
tribution is a novel global illumination algorithm which can concentrate computation on
important light transport paths and automatically adjust the energy distributed area for each
light transport path.
• Exploration of temporal coherence among paths:we extend the formulation of Markov
Chain Monte Carlo (MCMC) algorithms from rendering a staticimage to rendering an an-
imation. In addition to the formulation of animation rendering, a newtime perturbation
method is designed to mutate the path in the time domain to explore the temporal coherence
among path integrals. The time perturbation method is designed to reuse path samples with
similar properties at different moments of time in order to reduce the temporal artifacts of an
animation.
• A new lens mutation method: when we implement the MCMC algorithms, we found that
there is possibly high failure rate in the caustics perturbation. The high failure rate in the
caustics perturbation possibly comes from the bad prediction of the perturbation angle range
in the caustics perturbation. Thus, we purpose a new lens perturbation to increase the success
rate of perturbations .
5
• Two new regeneration methods:stratified exploration of the image plane is a limitation to
increasing rendering efficiency for the MCMC algorithms because the importance of regions
on the image are not perceptually the same. The ERPT and PMC-ER algorithms carefully
generate new paths according to the need of exploring the image plane evenly for achieving
unbiasedness. This choice is sub-optimal because some areas on the image plane contain
higher perceptual variance than others do. To contribute more computation effort in reducing
the variance in these high perceptual variance regions would increase the perceptual quality.
In addition, some types of paths such as caustics paths are visually important but hard to be
found with a general path tracing algorithm. Concentratingmore computation effort on these
paths can further improve the rendering efficiency. However, evenly exploring the image
plane prevents MCMCs from spending more computation effortin exploring those “hard-
to-find-but-important” paths and limits the improvement inrendering efficiency. And thus
two new path-regeneration methods to concentrate more computation on “high perceptual
variance” regions and “hard-to-find-but-important” pathsto enhance the rendering efficiency.
1.3 Thesis Outline
Chapter 2 gives an overview of Monte Carlo methods. After a brief history of Monte Carlo
methods, the principle of Monte Carlo integration, which uses Monte Carlo simulation to esti-
mate an integration, is described. We further introduce twoconcepts: Markov Chain Monte Carlo
methods and Population Monte Carlo methods. Both concepts are the core of this dissertation.
Chapter 3 introduces the basic concepts related to global illumination and physically based
rendering. After providing the definition of the most commonly used terms in radiometry, we
present the surface BRDF, the rendering equation, and the formulation for energy redistribution
with an extra dimension time. Then, a summary of existing Monte Carlo rendering algorithms for
rendering static images is given. Finally, we also give a survey of animation rendering algorithms
existing in global illumination community.
6
Chapter 4 presents Population Monte Carlo Energy Redistribution algorithms for rendering
static images. The algorithm concentrates computation on important light transport paths and
automatically adjusts energy distributed area for each light transport path based on the information
collected in previous iterations. The automatic adaptation eases the burden of setting parameters
for rendering a scene and also enhances the rendering efficiency.
Chapter 5 presents a new lens perturbation and new path-regeneration methods to improve the
rendering efficiency when using MCMC algorithms to render static images. The new lens per-
turbation simplifies the computation and control of the caustics perturbation method and possibly
increases the perturbation success rate. The new path-regeneration methods aim to concentrate
more computation on “high perceptual variance” regions and“hard-to-find-but-important” paths.
Chapter 6 presents a space-time algorithm that exploits thespatial and temporal coherence
among all paths based on the Markov Chain Monte Carlo framework when rendering a physically-
correct animation. A new perturbation technique calledtime perturbationis developed to explore
the temporal coherence among paths. Furthermore, in order to make animation rendering plausible,
we develop a parallel rendering system to distribute the iterative computational tasks to a pool of
computers. Each task is rendered with a set of task-based parameters without introducing bias.
Chapter 7 concludes with a summary and the original contributions in the thesis, and identifies
some future research directions.
7
Chapter 2
Monte Carlo Integration
Monte Carlo integration is the main tool used in this dissertation. In this chapter the basic
concepts of statistics related to Monte Carlo integration.We first present a simple history of Monte
Carlo. Following is the discussion of Monte Carlo integration. Metropolis-Hastings sampling
is the foundation for the Markov chain Monte Carlo (MCMC) algorithms in global illumination
which correlatedly generates samples in the integration space. Finally, we present Population
Monte Carlo which uses the information collected in the previous iterations to adjust the kernel
function to become a better approximation to the ideal sampling density in order to reduce the
variance. Good references for Monte Carlo methods include Kalos et al. [78], Hammersley et
al. [58], Spanier et al. [137], Robert et al. [122], Gilks et al. [49], Doucet et al. [35], and Liu [91].
2.1 Monte Carlo Methods: A Brief History
The first documented research about Monte Carlo happened in 1777. It was an experiment
conducted by Comte de Buffon for randomly dropping a needle of length L onto a board with
equidistant parallel lines. The distance between consecutive lines isd which d ≥ L. The experi-
ment showed that the probability of a needle intersecting one of these lines is
p =2L
dπ[2.1]
Later, Laplace showed that this can be used to estimate the value ofπ.
8
Modern Monte Carlo originated in the early 1940’s at the Los Alamos National Laboratory for
the need of developing nuclear weapons for World War II. In late 1946 Stainislaw Ulam suggested
the potential of using random sampling to simulate the motion of neutrons. With the development
of the first electronic computer, John von Neumann developeddetailed proposal for using Monte
Carlo as a research tool for developing nuclear weapons. Hisproposal leaded to small-scale simu-
lations whose results are important for the success of the nuclear-weapon project. Metropolis and
Ulam [96] published the first paper to describe their ideas inMonte Carlo. Their paper attracted a
great deal of researches in 1950’s. Metropolis also suggested the name “Monte Carlo” which is a
city in Monaco and famous for its casino.
2.2 Monte Carlo Integration
Standard numerical integration techniques have difficultyin solving the problems in high-
dimensional domains, especially when the integrand is not smooth. One of the most important
applications of Monte Carlo is to evaluate the integration of a function which is equivalent to eval-
uate the expectation of that function. In this section we would like to demonstrate how to transform
the computation of integration problems to the computationof expectation problems.
Assume that we would like to evaluate an integral of the form
I =∫
Ωf(x)dµ(x) [2.2]
whereΩ is the domain of integration which is a region in multiple-dimensional space,f : Ω→ℜ is the integrand and is a real valued function, andµ is a measure onΩ. In the basic form Monte
Carlo integration is to estimate the integral in Equation 2.2 by independently samplingN points,
X1, · · · , XN , according to some convenient density functionp(x), and then computing the estimate
IN =1
N
N∑
i=1
f(Xi)
p(Xi)[2.3]
In the original form we can chooseΩ = [0, 1]s wheres is the dimension of the integrand and
the samplesXi are chosen independently and uniformly at random. We can getthe estimator as
9
IN =1
N
N∑
i=1
f(Xi) [2.4]
It is easy to show that the expectation of the estimatorIN gives the correct result.
E[
IN
]
= E[
1
N
N∑
i=1
f(Xi)
p(Xi)
]
[2.5]
=1
N
N∑
i=1
∫
Ω
f(x)
p(x)p(x)dµ(x) [2.6]
=∫
Ωf(x)dµ(x) [2.7]
= I [2.8]
Assuming thatf(x)/p(x) is finite wheneverf(x) 6= 0. For estimating the variance of Monte
Carlo integration, we denoteCi = f(Xi)/p(Xi). We estimate the variance ofC:
V[
C]
= E[
C2]
−E[
C]2
=∫
Ω
f(x)2
p(x)dµ(x)− I2 [2.9]
Proposed thatV[C] is finite, we can observe that the variance ofIN decrease linearly withN :
V[
IN
]
= V[
1
N
N∑
i=1
Ci
]
=1
N2V[ N∑
i=1
Ci
]
=1
N2
N∑
i=1
V[
Ci
]
=1
NV[
C]
[2.10]
where we have used the facts thatV[
aC]
= a2V[
C]
andCi are sampled independently, and thus
V[
∑Ni=1 Ci
]
= NV[
C]
. Therefore, the standard deviation can be estimated asσ[IN ] = σ[
C]
/√
N .
Two important points can be drawn from the standard deviation derived above.
1. The standard deviation of Monte Carlo integration decreases at a rate ofN−1/2.
2. The statistical error is independent of the dimensionality of the integral. This is a very
important property because the numerical integration methods are well known to suffer the
curse of dimensionality i.e. the computational cost grows exponentially with the dimension
10
of the integrals. Thus, this property makes Monte Carlo integration a proper tool for solving
high-dimensional integration problems
There are several advantages of applying Monte Carlo integration. First, it converges at a
rate ofO(N−1/2) in any dimension, regardless of the smoothness of the integrand. This make it
suitable for global illumination problems because the integrand are often high-dimensional and
sometime not smooth. Second, it is easy to understand and simple to use. The only requirement
for using Monte Carlo integration is a density function,p(x), for generating samples and the ability
to evaluate the weight of a samplef(Xi)/p(Xi). Finally, it is general and flexible. They can be
applied to a wide range of problems. For example, in global illumination we can formulate the
rendering problems as path integrals in the path space of alllight transport paths. The domain of
the path space is naturally an infinite-dimensional space but it is straightforward to handle with
Monte Carlo integration.
2.2.1 Estimators and Their Properties
A functionF of random variables is called anestimatorfor an unknown quantityΘ if its mean
E[
F]
is a usable approximation ofΘ. A particular numerical value ofF after instantiating the
random variables with the known sample data, is called anestimate.
There are different possible estimators for a given quantity. Normally, we would like to evaluate
the performance of an estimator for a quality. In order to do the comparison among different
estimators, we need some criteria. In this section we presents some basic metrics to form the
criteria including mean squared error, bias, and efficiency.
• Mean Squared Error: to estimate the quality of an estimator is expressed by its mean
squared error (MSE). The MSE of an estimatorF to a quantityΘ is normally calculated as
MSE[
F]
= E[
(F −Θ)2]
[2.11]
which is the expectation of the square of the difference betweenF andΘ.
11
• Bias: F is called an unbiased estimator ofΘ if its expected value is exactly the same as the
true value ofΘ. If not, the difference between them is the bias:
β[
F]
= E[
F]
−Θ [2.12]
One advantage of using an unbiased estimator is that it is guaranteed to get the correct value
of Θ if enough samples are used. Also, the expected value of an unbiased estimator will be
the correct value after any number of samples, which makes itmuch easier to analyze the
error of the estimator.
• Efficiency: notice that we can make the variance of an unbiased estimator as small as pos-
sible by taking many samples. However, increasing the number of samples increases the
running time of the estimator. Thus, we would like to have an estimator which has small
variance and running time. The trade-off between running time and variance is denoted as
efficiencyof a Monte Carlo estimator [58]:
Eff[
F]
=1
V[
F]
T[
F] [2.13]
whereT[
F]
is the running time for the estimator andV[
F]
is the variance of the estimator.
However,we must point out that in some cases there may not exist any obvious choice of a
better estimator even some estimators can be clearly betterthan others in some other cases.
2.3 MCMC and Metropolis-Hastings Sampling
Since this dissertation is based on the Markov Chain Monte Carlo methods. In this section we
would like to provide the background knowledge for the Markov Chain Monte Carlo (MCMC)
method and Metropolis-Hastings sampling which is a well-know application of MCMC meth-
ods. MCMC methods apply Markov Chain simulation to generatesamples from a specific target
12
distribution,π(x) [49]. In other words, Markov chain Monte Carlo (MCMC) methods start up
with an initial sampleX0 and generate a random walk,X0, X1, X2, · · ·, through a distribution
Tr(Xz+1|Xz) called transition functionof the chain to approximate the target distributionπ(x)
in given state spaceΩ. The transition functionTr(Xz+1|Xz) is a conditional probability density
function and it depends on the current stateXz. From Monte Carlo’s point of view, the samples
generated by MCMC methods can be used to compute any expectation (or integrals) usingπ to an
acceptable degree of accuracy.
Metropolis-Hastings sampling [97, 59] provides a way to apply this simple concept to generate
such a Markov chain. To generate a sample, the algorithm starts with proposing acandidatestate,
X ′z+1 using the information ofXz and thetentative transition functionT (X ′
z+1|Xz) which is the
probability of going from stateXz to stateX ′z+1. The algorithm then either accepts the candidate,
X ′z+1 or retains in the original stateXz. Figure 2.1 shows the Metropolis algorithm:
1 InitializeX0; setz = 0.
2 for z = 1 to Nsample
3 X ′z+1 ← T (x|Xz)
4 generate a random numberr ∈ [0, 1]
5 if( r < A(X ′z+1|Xz) ) then
6 Xz+1 = X ′z+1
7 else
8 Xz+1 = Xz
Figure 2.1: The Metropolis sampling algorithm.
The functionA(X ′z+1|Xz) is theacceptance probability. It is computed as
A(X ′z+1|Xz) = min
1,π(X ′
z+1)T (Xz|X ′z+1)
π(Xz)T (X ′z+1|Xz)
[2.14]
13
Remarkably, tentative transition function can be almost any form to make the Markov chain
generated by the algorithm converge to the stationary distributionπ(x). Metropolis-Hastings sam-
pling is very general. The beauty of this algorithm is able tosample from any arbitrary complex
probability distribution function,π(x), as long asπ(x) can be evaluated. Although the purposal
distribution,T (Xz+1|Xz), can be almost any form for the chain to eventually converge to π(x), the
design of the proposal distribution has large impact on the convergence rate. Hence, the main step
for designing a Metropolis sampler is to find a good proposal distribution function.
2.4 D-Kernel Population Monte Carlo
The Population Monte Carlo algorithm [20] provides us an iterative importance sampling
framework. The distinguishing feature of PMC is that the kernel functions are modified after each
step based on information gathered from prior iterations. The kernels adapt to approximate the
ideal importance function based on the samples seen so far. While this dependent sampling may
appear to introduce bias, it can be proven that the result is either unbiased or consistent, depending
on whether certain normalizing constants are known (in our case they are known). The generic
D-Kernel PMC sampling algorithm [33, 34] which is an evolution of PMC is stated in Figure 2.2.
1 generate the initial population,s = 0
2 for s = 1, · · · , Niteration
3 adaptK(s)i (x(s)|x(s−1)
)
4 for i = 1, · · · , Npopulation
5 generateX(s)i ∼ K
(s)i (x|X(s−1)
i )
6 w(s)i = π(X
(s)i )/K
(s)i (X
(s)i |X(s−1)
i )
7 resampling process: elimination and regeneration
Figure 2.2: The generic D-Kernel Population Monte Carlo algorithm.
14
The algorithm works on a population of samples denoted by
X(s)1 , . . . , X
(s)N
, wheres is the
iteration number andN is the population size, to evaluate∫
D f(x)dx, where isf(x) = π(x)h(x)
by sampling according to the target distributionπ(x).
The algorithm first creates a set of initial population by using any unbiased sampling method.
A kernel function, K(s)i (x
(s)i |x(s−1)
i ), for each member in the population is adapted at the end of
the loop. The responsibility of the member kernel function is to take the existing member sample,
X(s−1)i , as input and produces a candidate new sample,X
(s)i , as output (line 5). The resampling
step in line 7 is designed to cull candidate samples with low weights and promote high-weight
samples. The resampling process consists of two steps: elimination and regeneration. It is designed
to eliminate the samples with low contribution to the final result and to explore new unexplored
regions. The weight computed for each sample,w(s)i , is essentially its importance weight. At any
given iteration, an estimator of the integral of interest can be computed and is unbiased forπ(h):
f(x) = π(h) =1
N
N∑
i=1
w(s)i h(X
(s)i )
E[
1
N
N∑
i=1
w(s)i h(X
(s)i )
]
=1
N
N∑
i=1
E[
w(s)i h(X
(s)i )
]
=1
N
N∑
i=1
∫
Ω
π(x)h(x)
K(s)i (x|x(s−1)
i )K
(s)i (x|x(s−1)
i )dx
=1
N
N∑
i=1
∫
Ωπ(x)h(x)dx
=∫
Ωf(x)dx
It concludes thatπ(h) is an unbiased estimator ofπ(h).
Before applying PMC to rendering problems, we must first answer the following questions:
• What is the sampling domain and how big is population size?
• What is the member function and what is the adaption criteria?
• What techniques are used for sampling from the kernel functions and resampling step?
In the following chapters we will answer them for our rendering algorithm.
15
Chapter 3
Basics for Global Illumination
The goal of global illumination algorithms is to simulate the light transport from the light
sources to the camera. There are complex interactions amongthe light and the scene surfaces
through the transport process. In this chapter we would liketo present the basic knowledge of
light transport needed for the later discussion in this dissertation. First, important physical quan-
tities used to describe the light transport are presented. Then, the light-scattering properties on
the scene surfaces are described. An analytical form of the light transport problem, the rendering
equation, is derived from the light-scattering events on the scene surfaces. It is very difficult to
find the analytical solution for the rendering equation. Therendering equation is transformed into
path integrals in light transport path space in order to use Monte Carlo integration. Several im-
portant ray-tracing Monte Carlo algorithms are reviewed and their strengths and weaknesses are
analyzed. Furthermore, the path integrals are highly correlated. Markov Chain Monte Carlo algo-
rithms can evaluate the path integrals correlatedly by sample reuse. Therefore, we reformulate the
path integrals to solve the rendering problem with Markov Chain Monte Carlo algorithms. Finally,
animation rendering algorithms in global illumination arereviewed. In the entire dissertation we
assume that the light speed is infinite. As a result the light transport events from a light source
to the camera happen transiently. In other words we describethe transient events happening at a
specific moment.
16
3.1 Radiometry
We discuss several important physical quantities for lighttransport in this section. All quanti-
ties are described as the distribution of energy with respect to one or more parameters.
• Flux Φ: energy per unit time i.e. the time rate of energy passing through a surface or a
region and is measured as Watts [W = J · s−1].
Φ =dQ(t)
dt[3.1]
whereQ is the total energy of the photon events. This quantity is normally used to describe
the rate at which energy is emitted, absorbed, or scattered by a finite surfaceS ∈ ℜ3. For
example, we can describe that a light bulb emits energy in a rate of 60 Watts which means
the total energy passing through an imaginary sphere surface surrounding the light bulb in a
rate of 60 Watts.
• Irradiance E: flux per unit surface area and is measured as [W ·m−2].
E(xt) =dΦ(xt)
dA(xt)[3.2]
wherext is a point on a surfaceS (either real or imaginary) and the superscriptt is used to
denote the moment when the measurement is taken anddA is the differential area around
the point with specific normalN(xt). This quantity is normally used to describe the mea-
surement of incident radiation on one side of surface (i.e light incident from the upward
hemisphere). For example, the irradiance from an un-occluded point light source at point,
xt, which is distancer away from the light isΦ · (4πr2)−1.
• RadianceL: flux per unit area per unit solid angle and is measured as [W ·m−2 · sr−1].
L(xt, ω) =d2Φ(xt, ω)
dA⊥ω (xt)dσ(ω)
[3.3]
17
wheredA⊥ is the projected area ofdA on a hypothetical surface perpendicular to the direc-
tion ω. Because the radiance is defined with “per unit solid angle”,the value of the radiance
does not attenuated. For example the radiance of a point light source atxt1 andxt
2 is the
same.
3.2 Bidirectional Scattering Distribution Function
The bidirectional scattering distribution function describes the light-scattering properties of a
surface. Let us consider the geometric relation shown in Figure 3.1.xt is a point on a scene surface
M and superscriptt denotes the moment of the event happening and we would like toconsider
the radiance leaving in a particular directionωo. The radianceLo(xt, ωo) depends on the radiance
arriving atxt from all directions and the properties of the surface. We first consider the outgoing
radiance for the incident light from an infinitesimal cone around the fixed directionωi where the
cone occupies a solid angle ofdσ(ωi). The irradiance generated by the light strikes at the pointxt
equals to
dE(xt, ωi) = Li(xt, ωi)dσ(ωi).
The light is then scattered by the surface in all directions.We represent the radiance leaving in
the direction ofωo from the incident light in the direction ofωi asdLo(xt, ωi). Experimentally, we
can observe a proportional relationship betweendE(xt, ωi) anddLo(xt, ωo) which is expressed as
dLo(xt, ωo) ∝ dE(xt, ωi)
As a result we define BSDF as the ratio of the outgoing radiancein the directionωo to the
incoming differential irradiance from the directionωi. It is a function of incoming direction, out-
going direction and surface pointxt:
fs(xt, ωo, ωi) =
dLo(xt, ωo)
dE(xt, ωi)=
dLo(xt, ωo)
Li(xt, ωi) cos θidσ(ωi)[3.4]
18
Figure 3.1: The input and output geometrical relation used to describe bidirectional scatteringdistribution function.
There are two important properties of BSDF. Models that havethese properties are considered
to be physically plausible.
• Helmholtz Reciprocity Rule: for any incoming and outgoing direction pair,ωi and ωo,
BSDF is symmetric to the directions:
fs(xt, ωo, ωi) = fs(x
t, ωi, ωo) [3.5]
• Energy Conservation Law: the energy conservation law says that the quantity of light
scattered must be less than or equal to the quantity of incident light. For any directionωo,
∫
Sfs(x
t, ωo, ω′) cos θ′dσ(ω′) ≤ 1 [3.6]
whereS is the imaginary hemispherical surfaace around the pointxt BSDF can be used to de-
scribe all possible relationship between incoming and outgoing direction. Physically, we separate
the scattering into two different events: reflection and transmission. As a result we split BSDF
into two different functions: bidirectional reflectance distribution function (BRDF) which is used
to describe the light-reflecting properties of a surface andbidirectional transmittance distribution
function (BTDF) which is used to describe the light-transmitting properties of a surface. By inte-
grating the scattered radiance of incident light
19
dLo(xt, ωo) = Li(x
t, ωi)fs(xt, ωo, ωi) cos θidσ(ωi)
overall directions, we can predictLo(xt, ωo).
Lo(xt, ωo) =
∫
S2Li(x
t, ωi)fs(xt, ωo, ωi) cos θidσ(ωi) [3.7]
This equation calledsurface scattering equationcan be used to predict the appearance of the
surface, given a description of the incident illuminance.
3.3 The Rendering Equation
In the past most photorealistically rendering research in global illumination is limited to render
static images. However, in this dissertation we would like to render an animation sequence. Thus,
we have to derive the rendering equation which is able to handle animation rendering from the
scattering function. Through the process, we will also define the data structure which is similar to
a scroll of film used in a movie camera to store the tempo-spatial samples to generate the animation.
Fig. 3.2 shows the conceptual diagram of the image sweep. Each frame in the animation is
stacked in the time-line aligning with the camera movement.To render a photorealstic image from
a scene must consider all the light interaction in the scene at the period of shutter open. In other
words the valid period of light transport events for a frame is betweenT shutterk andT shutter
k +
∆T shutter. The difference of denotation betweent andT is similar to the random variablex and
the random sampleX. In other wordsT is the specific moment for a sample andt indicates a
random time with a plausible moment in the entire animation.Here we simplified our discussion
by assuming that the period of shutter open is transient and thust = T for the following discussion.
Global illumination algorithms achieve photorealistically rendering by simulating light trans-
port behaviors based on physical principles. We start to consider the behavior of light transport by
first change the notation ofLo to Ls in Equation 3.7 to represent the outgoing radiance due to the
scattering behavior of the surface. The law of energy conservation predicts that exitant radiance at
20
Figure 3.2: The image sweep is a set of three dimensional hyper-cubes which correspond to theframes,Frame1, . . . , F rameN and formed by all image planes aligned with the movement of thecamera at different frame periods. Please notice thatk is used to index the frame andt is used toindicate the moment when a path is sampled. Valid path samples for framek are generated betweenT shutter
k andT shutterk +∆T shutter, whereT shutter
k is the moment when the shutter opens. In addition,all pixel samples in the valid period must also spatially pass through the image planeIt centeredat the center of the camera. In other words, the image sweep isthe sweep of the image plane alongthe camera’s locomotion in space along the time betweenT shutter
k andT shutterk + ∆T shutter.
21
a surface point along an outgoing direction must be equal to the sum of the emitted and reflected
radiances. We describe this relationship as theenergy balance equation:
Lo(xt, ωo) = Le(x
t, ωo) + Ls(xt, ωo) [3.8]
When we replace theLs term with the scattering Equation 3.7, we get
Lo(xt, ωo) = Le(x
t, ωo) +∫
Ωi
fs(xt, ωo, ωi)Li(x
t, ωi) cos θidσ(ωi) [3.9]
whereΩi is all possible input ray direction. Radiance along a ray is constant in free space i.e.
radiance will not change until it hit some surface or media. We define thext through a ray-casting
functionx′t = Tr(xt, ω) wherex′t is the first surface point visible fromx′t along the directionω.
The relationship between the incoming radiance and outgoing radiance can be written as
Li(xt, ωi) = Lo(Tr(xt, ωi),−ωi) [3.10]
Equation 3.9 can be rewritten as
Lo(xt, ωo) = Le(x
t, ωo) +∫
Ωi
fs(xt, ωo, ωi)L(Tr(xt, ωi),−ωi) cos θidσ(ωi) [3.11]
We obtain therendering equationfor generally describing the complex light transport behav-
iors inside a 3D animated or static virtual scene.
However, the goal of light transport is to compute a set of real-valuedmeasurementsI11 , · · · , I1
Npixel, · · · , Ik
1 , · · · ,for an animation orIk
1 , · · · , IkNpixel
for a static image. Each measurement corresponds to the output
of a hypothetical sensor that responds to the radianceLi(xt, ω) incident upon it. The response of
the sensor is characterized by the sensor response function, W kj (xt, ω). The overall response for a
pixel j of framek is determined by
Ikj =
∫ T shutterk
+∆T shutter
T shutterk
∫
M×SW k
j (xt, ω)Li(xt, ω)dA(xt)dσ⊥
xt(ω). [3.12]
22
whereM is the object surfaces in the scene, andS is the 3D imaginary hemispherical surface
above the sensor response surface. When rendering a static scene, the value ofk andt can be ne-
glected or set to zero. When rendering a frame with a transient shutter open period in an animation
scene i.e.∆Tshutter = 0, the value ofk andT will be a constant, andT can be computed fromk.
3.4 Monte Carlo Integrations for Global Illumination
The radiance arrives at pointxt from the directionωi can be calculated from the emitting and
scattering radiance from all surface points in the scene. Radiance from emitting depends on the
surface properties but the radiance from scattering depends on the incident and emitting radiance of
the surface points. In other words the rendering equation isa form of the Fredholm equation of the
second kind in which we need to solve the unknown quantityL appearing on the both sides of the
equation. The Fredholm equation is hard to be solved analytically. However, if we manipulate the
rendering equation mathematically to the formation of pathintegrals, the complex light transport
problem can be solved by using general-purpose integrationmethods.
3.4.1 Path Integral Formulation of the Rendering Equation
The goal of this section is to transform the rendering equation into a form of simple integrals.
In other words we would like to express the measurementIkj in each pixel of each frame in the
form of
Ikj =
∫
Ωfk
j (xt)dµ(xt) [3.13]
wherext is a valid light transport path at the momentt; Ω is the complete set of light transport
paths from a light source to the camera for the entire period when the scene exists;µ(xt) is the
surface area measure for the pathxt; fkj (xt) is the measurement of contribution to the pixelj at
framek from the pathxt.
23
Figure 3.3: Geometrical relation in three-point representation of thelight transport equation.
24
3.4.1.1 The Three-Point Formulation of the Transport Equation
The path is constructed from vertices and edges. Thus, it is easier for us to transform the light
transport equation into a three-point form. In other words we would like to replace the directional
variablesωi, ωo in the transport equation with verticesxt. Fig. 3.3 shows the geometrical relation
used in the three-point representation. First, the equilibrium radiance can be expressed in the form
L(xt → x′t) wherext,x′t ∈M are points on the surfaces. We can get
L(xt → x′t) = L(xt, ω) [3.14]
whereω = (x′t − xt)· ‖ x′t − xt ‖−1 is the unit directional vector fromxt to x′t. Then BSDF
can be expressed as
fs(xt → x′t → x′′t) = fs(x
′t, ωi, ωo) [3.15]
whereωi = (x′t − xt)· ‖ x′t − xt ‖−1 andωo = (x′′t − x′t)· ‖ x′′t − x′t ‖−1.
Finally, the light transport equation can be expressed in the three-point form
L(x′t → x′′t) = Le(x′t → x′′t)+
∫
ML(xt → x′t)fs(x
t → x′t → x′′t)G(xt ↔ x′t)dA(xt) [3.16]
whereG is the geometry term and can be represented as
G(xt ↔ x′t) = V (xt ↔ x′t)‖ cos θo cos θ′i‖‖ xt − x′t ‖ [3.17]
whereV is the visibility test, its value is 1 ifx andx′ mutually visible.
We can also express the measurement Equation 3.12 as
Ikj =
∫ T shutterk
+∆T shutter
T shutterk
∫
M×MW k
j (xt → x′t)L(xt → x′t)G(xt ↔ x′t)dA(xt)dA(x′t). [3.18]
25
3.4.1.2 The Path Integral Formulation
In order to use general-purpose integration methods, we would like to express each measure-
ment in the path-integral form
Ikj =
∫
Ωfk
j (xt)dµ(xt)
We useΩl to represent the set of paths of lengthl, i.e. the set of paths have the form
xt = xt0x
t1 . . .xt
l
where1 ≤ l ≤ ∞ andxtl ∈ M for eachl and each vertexxt
l is the intersection of the light
transport on the surface at the momentt. We definearea-product measureµl on this set of paths
as
µl(D) =∫
DdA(xt
0) . . . dA(xtl)
whereD ⊂ Ωl is a set of paths. Formally,µl is an area product measurement.µl can also be
defined as
dµl(xt0 . . .xt
l) = dA(xt0) . . . dA(xt
l) [3.19]
Next, the path spaceΩ can be defined as
Ω =∞⋃
l=1
Ωl [3.20]
i.e. Ω represents the set of paths with finite lengths. And the area-product measurementµ to
this space is
µ(D) =∞∑
l=1
µl(D⋂
Ωl) [3.21]
26
We finish the formulation of path integrals by recursively expanding the transport Equation 3.16
to obtain
Ikj =
∑∞l=1
∫
Ml+1 Le(xt0 → xt
1)G(xt0 → xt
1)l−1∏
i=1
fs(xti−1 → xt
i → xti+1)G(xt
i → xti+1)
·W kj (xt
l−1 → xtl)dA(xt
0) · · · dA(xtl)
=∫
M2 Le(xt0 → xt
1)G(xt0 → xt
1)Wkj (xt
0 → xt1)dA(xt
0)dA(xt1)
+∫
M3 Le(xt0 → xt
1)G(xt0 → xt
1)fs(x0t → xt1 → xt
2)G(xt1 → xt
2)
W kj (xt
1 → xt2)dA(x0)dA(x1)dA(x2)
+ · · · [3.22]
The integrandfkj for different path lengths can be defined by expanding the appropriate term
in Equation 3.22. For example,fj for a pathXT = XT0 XT
1 XT2 , we have
fkj (Xt) = Le(X
T0 → XT
1 )G(Xt0 → XT
1 )fs(XT0 → XT
1 → XT2 )G(XT
1 → XT2 )W k
j (XT1 → XT
2 )
This functionfkj is called themeasurement contribution function. Now we finish defining all
the terms needed for path integrals.
There are several benefits to have the path integral model. First, by reducing light transport to an
integration problem, it allows general-purpose integration methods to be applied. Second, the path
integrals also lead to new techniques for sampling paths locally because it describes scattering from
one surface at a time. This allow light transport algorithmsto construct a path incrementally by
recursive sampling of the integral equation. Finally, the path integral model is useful to understand
the limitations of unbiased Monte Carlo algorithms.
3.4.2 Monte Carlo Integrations
To render a frame in an animation is to compute the intensity,Ikj , of each pixelj at each frame
k, by estimating the integrals:
27
Ikj =
∫
ΩW k
j (xt)L(xt)dµ(xt), [3.23]
whereΩ is the path space,W kj (xt) is the measurement function for pixelj of k-th frame – non-
zero if xt passes through the image plane at a pixel position within thesupport of the reconstruction
filter at pixel j of thek-th frame – andL(xt) is the radiance carrying by light transport pathxt.
Note thatW kj is dependent on the pixel location and the moment of the path and thus, it depends
only on the last edgextl−1x
tl of the path. We are ignoring, for discussion purposes, depthof field
effects, which would necessitate integration over directions out of the pixel. Taking an importance
sampling view, given a set of samples,
XT11 , . . . , XTN
N
from an importance functionp(xt) for a
animation, the intensity in each pixel is estimated using
Ikj =
1
N
N∑
m=1
W kj (XTm
m )L(XTmm )
p(XTmm )
[3.24]
According to Monte Carlo integration,Ikj = Ik
j when the number of samples taken approaches
infinity. We note that the only difference among all pixels isthe term ofW kj (·). Equation 3.24
can be broken into many integrals, one for the support of eachpixel. Providedp(XT ) is known in
each path which passes through the valid image plane, the global nature ofp(XT ) is not important.
The rendering equation can be transformed to evaluate the expected radiance,E[
L(XT )]
, carried
by each sampled path and then accumulate this radiance in pixels whose support of reconstruction
includes the path-passing pixel position. At the final step of rendering, the accumulation in each
pixel is averaged by the total number of samples dropped in the support of the pixel.
The MC algorithms discussed in Section 3.4.3 are used to render a static image and thus we
can sett = 0. In additions since they also neglect the motion blur effectwhich is an integration
of all path contribution in the time domain when shutter opens i.e. ∆T shutter = 0. Under this
assumption we can set the time of transport event at a transient momentTk.
28
3.4.3 Raytracing-Based Monte Carlo Rendering Algorithms
There are many Monte Carlo algorithms developed to solve therendering equation.Path trac-
ing is the first Monte Carlo algorithm introduced by Kajiya [77] to solve the rendering equation
unbiasedly. In this section we summarize several importantMonte Carlo raytracing algorithms in-
cluding path tracing, bidirectional path tracing, irradiance caching, and photonmapping. The first
two algorithms are unbiased and the last two algorithms are biased.
3.4.3.1 Path Tracing
James Kajiya first presented the rendering equation in the paper [77] and then provided path
tracing as the solution to the rendering equation. Path tracing is the first algorithm to find the
complete solution for global illumination. Path tracing generates a random ray trees starting from
the camera. Each branch of the tree is a valid path by startinga ray from the camera, tracing
the ray through the scene and finally reaching a light source.At each bounce, a new direction
for the ray is sampled according to a distribution function corresponding to the surface properties
at the interaction point. The contribution of each valid path to the image plane is evaluated by
the radiance the path carries weighted by the probability ofthis path being generated. Each valid
path is then viewed as a valid sample for a pixel if the sensor function of the pixel cover the pixel
position which the path passes through. At the end the average of the contribution radiance from
all sample paths passing through the sensor located on the pixel is the measurement of intensity of
that pixel.
A variation to the path tracing algorithm starts the tracingray from a light source instead of the
camera. The algorithm is known as light tracing, particle tracing or backward ray tracing. Light
tracing is a dual algorithm of path tracing because the physics of light transport does not change
when a path is reversed according to the Helmholtz reciprocity rule. Both path tracing and light
tracing have their own advantages and disadvantages. Furthermore there are algorithms combin-
ing them to overcome the limitations of path tracing and light tracing and become a more robust
algorithm. The combination algorithms are called bidirectional path tracing, which is discussed
next.
29
3.4.3.2 Bidirectional Path Tracing
Lafortune [84] and Veach [150] developed bidirectional path tracing independently. However,
their algorithms are based on different statistic frameworks. Veach proposed the bidirectional
importance sampling algorithm based on the framework of multiple importance sampling while
Lafortune formulated the bidirectional path tracing algorithm as a recursive evaluation of the global
reflectance distribution function (GRDF).
Bidirectional path tracing generalizes the standard path tracing algorithm. When observing
paths generated by path tracing and light tracing, some types of sub-paths are more efficiently
sampled when tracing from the camera and some types of sub-paths are more efficiently sampled
when tracing from light sources. Thus, combining both typesof tracings can result in a better
tracing algorithm. Bidirectional path tracing first generates a pair of “eye” sub-path which starts
tracing from the camera and “light” sub-path which starts tracing from light sources. Multiple
complete paths can be formed by connecting the different vertices in the pair of “eye” sub-path
and “light” sub-path. Multiple importance sampling [151] weighs the contribution of these full
paths to form the estimator of pixel intensities. Multiple importance sampling provides a set of
near-optimal weights to reduce the variance of the estimator.
Bidirectional path tracing combines the advantages of pathtracing and light tracing. It is un-
biased and can handle a scene with arbitrary geometry and lighting. Because it link the “eye” and
“light” sub-paths, it provides the advantage of combining both the visual importance from “eye”
sub-paths and the lighting importance from “light” sub-paths. It significantly reduces the variance
in a rendered image compared to path tracing. However, it still requires a huge amount of samples
to generate a smooth image. Because “eye” and “light” sub-paths have to be connected to form
valid full paths, bidirectional path tracing is not suitable for scenes where most “eye” and “light”
sub-paths are not visible to each other. For example, the eyeis in one room which is different from
the room where the light source is and the connections among “light” and “eye” sub-paths are
through a small door jar. Most of the connections probably fail. In addition, each path is generated
independently. The important path information has been thrown away when generating a new path.
30
As a result even a difficult and important path may be located but the information related to this
path is thrown away when it generates the next sample path.
3.4.3.3 Irradiance Caching
Indirect lighting usually changes slowly on diffuse surfaces. Thus, we can first compute and
store the accurate computation of the indirect lighting at asparse set of scene locations. The stored
computations can be used to estimate the indirect lighting at new locations. This is the basic idea
of irradiance caching [169].
The detail of the algorithm is as follows. When estimating exitant radiance at a diffuse point,
the algorithm looks up the irradiance cache for one or more acceptable samples. The gradients
of the irradiance [168] estimated from cached samples are used to indicate the acceptability of
the cached samples. This method considers the distance of samples and the irradiance difference
caused by the changing in position and orientation. If the algorithm finds one or more acceptable
samples, an interpolated irradiance value is calculated from those samples; otherwise, the algo-
rithm estimates the accurate irradiance at that point and stores the value into the irradiance cache.
Since the algorithm only caches the irradiance values, the information on the directional distri-
bution of the incoming radiance is lost, and so this technique can only be used for diffuse surfaces.
In addition, the algorithm uses interpolation of cached irradiance which introduces bias into the
result. Therefore, advance convergence tests cannot be applied to this algorithm.
3.4.3.4 Photon Mapping
Photon mapping [73] considers the advantages of tracing paths from the camera and light
sources and caching the indirect lighting in irradiance caching. It uses a two-pass rendering pro-
cess. The first pass shoots photons from light sources by using standard light tracing. Whenever
a photon intersects a non-specular surface (diffuse or glossy), caches calledphoton mapsstore a
record of the position, incoming direction, the flux of the photon according to the photon proper-
ties. The second pass traces view rays from the camera to render the image by using the photon
31
maps built in the first pass. The photon mapping algorithm divides the integrand into four com-
ponents: direct lighting, specular reflection, caustics, and indirect lighting. Direct lighting and
specular reflection are accurately evaluated using standard Monte Carlo ray tracing. The caustics
is evaluated via a caustics photon map. The indirect lighting is evaluated via an indirect photon
map.
Photon mapping is reasonably efficient to handle all lighting effects such as caustics and color
bleeding. Another advantage is that the photon map does not depend on the underlying scene
geometry, which means that it scales well with scene complexity. However, the estimation of radi-
ance introduces bias into the algorithm. This limits the algorithm from using advance convergence
tests and numerical evaluation of the performance of the algorithm.
3.5 Markov Chain Monte Carlo Algorithms for GlobalIllumination
All algorithms discussed in the previous section generateseach sample independently. Thus,
important path information will be lost through the generation process. In addition, the path inte-
grals are highly correlated. Thus, sample reuse is an important technique to reduce the rendering
variance. In this section we would like to use Markov Chain Monte Carlo algorithms for sample
reuse. In the following discussion we use the variable superscript t to denote the moment when
the path is created. In static image renderingt is fixed to a constant,T . In animation renderingt
can be any moment of time in the entire animation and this denotation allows us to do the muation
in time for exploring temporal coherence among light transport paths. We will explain the time
mutation in Ch. 6.
3.5.1 Formulation for Markov Chain Monte Carlo Algorithms
Eq. 3.23 split the contribution functionfkj into two components, the measurement function of
the sensor located at pixelj of framek, W kj , and radiance carried by the path,L. The radiance is
independent ofW kj and thus can be reused for all pixels. Therefore,b =
∫
Ω L(xt)dµ(xt) expresses
32
the total radiant energy delivered through the image sweep as showed in Figure 3.2. The idea of
Markov Chain Monte Carlo algorithms is to generate a sequence of paths,XT00 , · · · , XTN
N , accord-
ing to the contribution of the paths to the image sweep, whereeachXTzz is obtained by a random
mutation to the pathXTz−1
z−1 . Through the process paths mutate, and newly mutated paths are ac-
cepted or rejected with a carefully chosen probability to ensure that paths are sampled according
to their contribution to the ideal image sweep. The mutations can have almost any desired form,
and typically involve adding, deleting, or replacing a small number of vertices on the current path.
In other wordsp = (1/b)L is implicitly used as the density function to generate a sequence of
samples. The integration can now be evaluated as
Ikj = E
[
1
N
N∑
i=1
bW kj (Xi
Ti)]
3.5.1.1 Acceptance Probability
The Metropolis algorithm constructs a transition functionTr whose resulting stationary dis-
tribution is proportional to the givenL, and which converges toL as quickly as possible. Time
mutation will be discussed in Ch. 6. GivenXTz−1
z−1 , we obtainedXTzz as follows. First, we choose a
tentative sampleX′Tz′z , which can be done in almost any way desired. Generally the process can be
represented by thetentative transition functionT (·|·), whereT (YTY |XTX) gives the probability
density thatX′T ′
zz = YTY given thatXTz−1
z−1 = XTX . The tentative sample is then either accepted or
rejected, according to an acceptance probabilityA(YTY |XTX). To see how to setA(YTY |XTX),
suppose that we have already reached equilibrium, i.e.pi−1 is proportional toL. We must define
the transition function,Tr(YTY |XTX), such that the equilibrium is maintained. To do this, con-
sider the density is proportional toL(XTX )T (YTY |XTX)A(YTY |XTX ), and a similar statement
holds for the transition density fromYTY to XTX . To maintain equilibrium, it is sufficient that
these densities be equal:
L(XTX )T (YTY |XTX )A(YTY |XTX) = L(YTY )T (XTX |YTY )A(XTX |YTY ) [3.25]
33
This condition is known asdetailed balance. Thus the unique equilibrium distribution must be
proportional toL. In order to reach equilibrium as quickly as possible, the best strategy is to make
A(YTY |XTX) andA(XTX |YTY ) as large as possible which is achieving by letting.
A(YTY |XTX ) = min
1.0,L(XTX )T (YTY |XTX )
L(YTY )T (XTX |YTY )
[3.26]
3.5.1.2 Hybrid Algorithms Combining Independent Monte Carlo Samplingand Correlatedly Markov Chain Monte Carlo Sampling
Veach [149] proposed that using importance energy,f/p, of the seed path to weight each
mutation of a Markov Chain can remove the start-up bias if thedetailed balance is maintained at
each mutation. Therefore, a hybrid algorithm, energy redistribution with balance energy transfer,
is proposed to combine the independent sampling path tracing algorithm with the correlatedly
sampling Metropolis algorithm. The idea is to generate a setof independent samples by path
tracing as seed paths. Then each seed path is used to correlatedly explore a local path space,
Ωxt, around the seed path by Metropolis.Ωxt is a sub-space ofΩ and is defined as the domain
of all paths which can be reached through a sequence of perturbations from the seed path. This
definition is used to guarantee that the local exploration can reach and stay at the local stationary
probability,f(xt)/bxt wherebxt =∫
Ωxt
f(xt)dµ(xt) ≤ b. The estimation of the pixel intensity can
be calculated as
Ikj =
1
NMC
NMC∑
z=1
1
NMCMC
NMCMC∑
l=1
f(XTzz )
p(XTzz )
W kj (XTzl
zl )
[3.27]
The expectation of the estimation can be calculated as
E(Ikj ) =
∫
Ω
f(xt)
p(xt)E
1
NMCMC
NMCMC∑
l=1
W kj (XTl
l )|xt
p(xt)dµ(xt)
=∫
Ωf(xt)
∫
Ωxt
W kj (yt)
f(yt)
bxt
dµ(yt)dµ(xt) [3.28]
whereE(·|xt) is the conditional expectation on an independent seed pathxt.
Before solving Equation 3.28, we observe that if two pathsXT11 andXT2
2 are mutually reachable
by a sequence of perturbations, the local path domain based on XT11 is equal to the local path
34
domain based onXT22 i.e. Ω
XT11
= ΩX
T22
. This is true because any pathxt ⊂ ΩX
T11
can be reached
from pathXT22 by mutatingXT2
2 to XT11 by a sequence of perturbations and then toxt. In other
wordsxt is also inΩX
T22
. Similarly, any pathxt ⊂ ΩX
T22
is also inΩX
T11
. Therefore, we can define
a maximal pertubation path domain,Ωp, as a set of all paths that are reachable by a sequence
of perturbations from any member in the domain and accordingto our discussion we can get
Ωp = ΩX
T11
= ΩX
T22
. The entire path domain,Ω, can then be partitioned asΩp,1⋃
Ωp,2 · · · where
Ωp,m is a maximal perturbation path domain andΩp,m⋂
Ωp,n = 0 wherem 6= n. The expectation
in Equation 3.28 can now be written as
E(Ikj ) =
∑
i
∫
Ωi
f(xt)
bp,i
dµ(xt)∫
Ωp,i
W kj (yt)f(yt)dµ(yt)
=∑
i
∫
Ωp,i
W kj (yt)f(yt)dµ(yt)
=∫
ΩW k
j (yt)f(yt)dµ(yt)
wherebp,i =∫
Ωp,if(xt)dµ(xt). Therefore, the estimator is unbiased. The variance analysis in [9]
allows us to bound the variance of the Markov Chain of a seed path as
VMCMC(Xi) ≤1
NMCMCbXi
NpixelEkj (1 +
2R(1)
q(1− q)−R(1))
whereNpixel is the number of pixels,Ekj is the average importance energy of the path associated
with this pixel,q is the probability that path contributes to the pixel under considerations, andR(1)
is the correlation between random variance indicating thattwo subsequent paths go through the
same pixel. Each seed path is generated independently and thus the variance of the entire hybrid
estimator is equal to the sum of variance in each Markov Chainand can be calculated as
V C(Ikj ) =
1
N2MC
×NMC ×1
N2MCMC
NMCMC∑
l=1
VMCMC(XTi
i )
≤ 1
NMC ×NMCMCbNpixelE
kj (1 +
2R(1)
q(1− q)− R(1))
Thus the variance of the entire algorithm isO(1/NMC), and the algorithm converges to the correct
answer if the number of independent samples go to infinity.
Veach [149] also proposed that we can use an equal weighting scheme if we resample the paths
according to its importance energy. Thus, we can use a similar deterministic sampling strategy
35
in [40] to generate total number of∑
Ei(XTi
i )/E paths whereEi(XTi
i ) =f(X
Tii
)
p(XTii
)andE is the aver-
age path energy and equal to∑
Ei(XTi
i )/N whereN is the number of paths. In other words, each
seed path has a number of Markov Chains proportional to its energy. The equal weighting scheme
now becomes energy redistribution with equal deposition and the estimator can be calculated as
Ikj =
E
NMC
NMC∑
z=1
Npath∑
m=1
1
NMCMC
NMCMC∑
l=1
W kj (XTzml
zml )
[3.29]
whereNpath = Nfloor + (0 : 1?E(XTi
i ) − (Nfloor + U(0, 1)) × E > 0) andNfloor = ⌊E(XTii
)
E⌋.
We can observe thatE(ENpath) = E(XTi
i ) =f(X
Tii
)
p(XTii
)which is the importance energy of the in-
dependent sampling algorithm for the path. The expectationand convergence analysis is similar
to the one discussed in the previous paragraph. Two criteriaare needed to maintain the unbi-
asedness of energy redistribution with equal deposition: first, the detailed balance of mutation is
maintained through the process for reaching and staying in the stationary probability; second, the
number of Markov Chains is proportional to the energy carried by the path for keeping the in-
dependent sampling algorithm unbiased. Later, we will showthat although our kernel function
for each path adapts from iteration to iteration, it is fixed inside a single Markov Chain. In ad-
dition, the acceptance probability is chosen to maintain the detailed balance at each perturbation
and thus the stationary probability requirement is achieved which implies convergence and unbi-
asedness. Furthermore, our algorithm deposits the remaining energy of an eliminated path by the
equal deposition method back into the image sweep before a path is removed from the population
in resampling process or terminated at the end of a task. Thisachieves the second criterion of
unbaisedness.
3.5.2 Markov Chain Monte Carlo Global Illumination Methods
The previous section discusses the fundamental theory of the Markov Chain Monte Carlo algo-
rithms. In this section we would like to review two importantalgorithms related to this dissertation:
Metropolis Light Transport (MLT) and Energy Redistribution Path Tracing (ERPT)
36
3.5.2.1 Metropolis Light Transport
Metropolis Light Transport (MLT) is a rendering algorithm based on the Metropolis sampling
framework [149]. Metropolis sampling samples paths according to their contribution to the image
plane by means of Markov Chain Monte Carlo random walks. A sequence of dependent paths
are generated through a well-defined transition function toreach the stationary distribution of that
chain which is proportional toL. It only requires thatL is known up to a constant scale and can
be evaluated at each point in the domain. In other words, no analytical form forL is necessary.
The mutation strategies proposed in the algorithm achieve the ergocity and local exploration of the
path space and generate good results.
The key advantage of MLT is to coherently explore the path space. As a result, once an im-
portant path is found, the relative path space around this important path will be explored. It is
especially good for the difficult but important paths. As a result, MLT is very efficient in handling
traditionally difficult scenes such as light going through adoor jar. Another advantage of MLT is
that the Metropolis sampling framework ensures its unbiasedness. MLT is also competitive with
previous unbiased algorithms for relatively simple scenes.
3.5.2.2 Energy Redistribution Path Tracing
Energy Redistribution Path Tracing (ERPT) introduced by Cline et al. [25] is a hybrid global
illumination algorithm that combines the ideas of metropolis light transport and path tracing. ERPT
can be understood as multiple seed paths MLT algorithm: the algorithm starts by a set of initial
paths passing through stratified pixel positions on the image plane. The initial paths then are used
as a seed path for starting the Markov Chains. As in MLT, a tentative path is generated by mutating
the original seed path and accepted with a probability to maintain thedetailed balancefor the
chain. The path mutation strategies are to locally explore the correlated paths in the path space
to reduce the rendering variance. The set of initial paths generated by the standard path tracing
algorithm is used to guarantee the ergocity. However, this algorithm uses post-processing noise
filters to reduce image noise; this introduces bias.
37
3.6 Population Monte Carlo Rendering
Fan in his dissertation [40] proposed three physically-based rendering algorithms, Population
Monte Carlo Image Plane (PMC-IP), Population Monte Carlo Hemispherical Integrals (PMC-HI),
and Population Monte Carlo Path Tracing (PMC-PT), which arebased on the Population Monte
Carlo framework. PMC-IP uses the variance of sample path radiances in each pixel to adaptively
decide the pixel-sample distribution function on the imageplane. The algorithm distributes more
samples to regions with high perceptual variance in order toreduce the variance in the cost of
increasing noise in smooth regions. However, little increment of noise in the smooth regions can
improve the perception of the entire image by reducing the variance in the high perceptual variance
region. PMC-HI uses the variance of direct lighting sampleson the estimated surface point to adjust
the sampling methods. PMC-IP and PMC-HI can easily cooperate with ray-tracing algorithms such
as path tracing and photonmapping. However, the PMC-HI algorithm cannot be used in Markov
Chain Monte Carlo algorithms because MCMC algorithms directly use a light vertex instead of
the estimation of direct lighting. Although PMC-IP inspired us to concentrate more computation
effort on the visually important areas on the image plane, directly applying the uneven exploration
of the image plane into energy redistribution introduces bias into the final result. Therefore, we
design our new variance-regeneration method with a proper weighting scheme based on the PMC-
IP framework in order to prevent bias. In addition, the adaptation in both algorithms is still in a
frame-by-frame manner.
In the initial phase PMC-PT traces a set of paths such as 3 samples per pixel (SPP) with a gen-
eral path tracing algorithm and then randomly chooses part of the paths as population paths. In each
iteration, PMC-PT perturbs the population paths with lens perturbation to explore the spatial coher-
ence among paths. The radiance of the newly perturbed path iscomputed byf(ys)/K(ys)|ys−1).
The radiance is labeled with the chosen perturbation radius. In resampling phase, the accumulation
of the labeled radiances is used to adapt the kernel functionand eliminate part of the population.
From our observation we find that there are several limitations: First, PMC-PT only uses part of
path-tracing samples. This wastes a large amount of computation effort because the cost to trace
38
a path is very high. In our algorithm we try to distribute the energy in each generated path to the
surrounding pixels to prevent the cost of extra path tracing. Second, the resample criterion used in
PMC-PT normally concentrates too much computation effort on exploring the high-contribution
paths. Our PMC-ER algorithms adapt the PMC framework into the energy redistribution algorithm
and use the energy of a path to determine the life time of the path in the population. In addition,
the acceptance probability is used as the criterion for the adaptation of mixture probability because
the acceptance probability is a better indication of how well the chosen perturbation performed
Third, PMC-PT uses a single mixture function for the entire population. Although the adaption
for the entire population can adjust the perturbation parameters according to the properties of the
scene, the difference among paths is large even in the same scene. Thus, the population path in our
algorithm uses an individual kernel function to consider the difference in properties of each path.
Finally, PMC-PT only uses lens perturbation methods to explore the correlated paths. However, as
we will discuss in Ch. 5, the failure rate of lens perturbations is hundred percent when applying the
lens perturbation to a caustics path. Thus, PMC-PT performslike a path tracing algorithm when
tracing a caustics path and this introduces larger bright spots on the image plane and affects the
performance of the algorithm. We improve the algorithm by developing a new lens perturbation
algorithm for enhancing the rendering efficiency.
3.7 Global Illumination Animation Rendering
Since the goal of this dissertation is to enhance the efficiency of animation rendering. Thus,
in this section, we review animation rendering algorithms in the global illumination community.
Damez et al. [27] presented a complete survey of research on animation rendering in the global
illumination community. Interested readers can refer to itfor further information.
39
3.7.1 Hybrid Methods of Image-Based Rendering and Global Il-lumination
An approach proposed by Nimeroff et al. [107] is to combine image-based approaches with
radiosity methods to render a global illumination walk-through. A set of precomputed images is
used to represent the 3D space of the environment. The set of images is constructed in such a way
that any viewpoint inside the environment can be generated through an interpolation process of
these images. Each precomputed image represents the directand indirect illumination computed
by using the standard wavelet radiosity. The algorithm is efficient but it is an open problem to find
an optimum set of images to represent the animated scene and also everything inside the scene
except the view point must be static and the material must be diffuse.
Myszkowski et al. [104] used a hybrid algorithm combining ray tracing and image-based ren-
dering (IBR). The algorithm derives as many pixels as possible by using inexpensive IBR tech-
niques without affecting the walkthrough animation quality. They also proposed a perception-
based tempo-spatial animation quality metric which is designed specifically for handling synthetic
animation sequences.
Yee et al. [172] demonstrated that greater errors can be tolerated for less salient image regions
in which the density of indirect lighting samples can be substantially reduced. They proposed to
compute the salience map which is used to control the irradiance caching for secondary lighting.
The global illumination solution is computed for a set of key-frames and then interpolation is used
to compute the inbetween frames. Such an approach might result in visually noticeable errors
in the lighting distribution, which is affected by changes in the environment in the course of the
animation. [172, 104] face the same problem: the accuracy oflighting reconstruction fluctuates
between frames, achieving the highest level for the key-frames, and then gradually decreasing for
in-between frames.
40
3.7.2 Radiosity Methods
The radiosity community has conducted a significant amount of research on animation ren-
dering. First, research attempted at efficiently using radiosity methods to compute high-quality
animation focused on adapting the full-matrix and progressive radiosity algorithm to take advan-
tage of time coherence. Baum et al. [11] identified form factors between patches that do not interact
with moving objects in preprocess. Modified form factors areincrementally computed by using a
reprojection technique which are to project the bounding box of the moving objects on hemicubes
to determine the form factors that need to be updated. Later,[108, 37, 38] proposed mechanisms
for the prediction of changes in form factors based a geometrical data structures call visibility com-
plex and visibility skeleton. [115] proposed to follow the principle of cell animations and compute
every frame as the sum of a fixed image and a frame-dependent one. However, the requirement of
huge amount of memory and lack of scalability prevent them from further development.
There exist two major groups of methods: progressive radiosity uses an incremental compu-
tation of the radiosity solution based on the previous frameand relies on shooting out corrective
energy to the scene regions affected by the environment changes [23, 47, 100, 99] and hierarchical
radiosity updates the hierarchy of links between the staticobjects affected by moving occluders
using different clustering methods [44, 132, 36, 126, 92, 28]. Those algorithms introduce mecha-
nisms for controlling the frame rate and efficiently identifying which part of the scene is modified.
However, the hierarchical radiosity framework poorly supports the light transfer between glossy
and specular surfaces. This drawback can be eliminated by adding photon tracing atop of the
line-space hierarchy, used for diffuse surface. The photons traversing changed scene regions are
identified using a dynamic spatial data structure and are selectively updated. But still, the memory
requirements in all those hierarchical approaches are extremely high due to the storage of the link
structure.
In Global Monte Carlo Radiosity [14, 15] the temporal coherence of costly visibility compu-
tations is efficiently and conservatively exploited by using the same global lines for the whole
animation, but then the radiosity solution is performed independently for each frame. Global lines
are tested for intersection once against all static objects, and against every frame position of the
41
dynamic objects. Visibility lists are built for each globalline. Static and dynamic lists are then
merged and sorted for every frame and used to perform the light transfer. Only visibility is reused
not the radiosity. In addition, limitation to nearly diffuse material makes them have limited usage.
Recently a two-pass radiosity method was proposed by Martinet al. [93]. Their algorithm
takes advantage ofa priori knowledge of the dynamic properties of the scene to exploit temporal
coherence. Their algorithm first runs a global pass to identify the “bad” and “good” links. The
second pass then updates the “bad” links according to the movement of objects. However, as
mentioned previously, radiosity techniques are limited bythe ability to handle general scattering
models and general geometric representations.
3.7.3 Virtual Light Sources
[82] implemented a hybrid method which consisted of two steps: first, CPU traces the photons
shooting out from light sources into the scene to construct aset of virtual light sources; second,
a multiple-pass hardware rendering techniques are appliedto render each virtual light sources. In
a dynamic environment, the virtual light sources are eliminated according to certain aging criteria
and added into the pool to explore the temporal coherence. [156, 160] added the ability of handling
more general lighting effects into the instant radiosity algorithm. In addition, they implemented the
algorithm in a computer cluster to achieve the interactive rate but each frame is rerendered inde-
pendently and the temporal coherence is not explored. [158]extended the architecture discussed
in [156, 160] to solve the multiple light sources problem by creating a cumulative distribution
function in a preprocess phase and using the CDF to sample important light sources. The temporal
averaging CDF is used to let virtual light sources generatedfrom the same light sources as tem-
poral consistent as possible. However, the number of VPLs from each light source may change
from frame to frame. The temporal artifacts still exist. [128, 129] used bidirectional path tracing
and metropolis light transport to distribute the virtual light sources in the scene to make sure that
virtual light sources appear in visually important areas. However, these two algorithms focused
on the single frame rendering. They do not consider any temporal coherence. [87] employed a
42
two-pass instant radiosity method to reduce the temporal artifacts. In the first pass, a set of one-
bounce virtual light sources are created and the corresponding shadow maps are created. And in
the second pass hardware local lighting is calculated at each visible surface point from the camera
with the depth information in the shadow map. For the following frame, part of the light sources
are removed and some new light sources and shadow maps are added into the structure. The re-
mained light sources can enhance the temporal coherence among rendering results. However, the
invalidity of the remained light sources introduces bias and the algorithm is limited to a scene with
diffuse or slightly glossy surfaces. [166, 163, 65] solved the many light sources problem by clus-
tering the contribution of light sources to a set of representative point light sources and computed
the surface appearance from the representative light sources in a static scene. In the preprocess,
a set of virtual light sources is created accordingly. The algorithms are for static rendering and
thus there exists serious temporal flickering. [66] solved the temporal flickering by using a tensor
clustering method to do the clustering of point light sources in spatial and temporal domain at the
same time. However, all these light-clustering methods areapproximations. They are approximate
to real answers and can provide fast feedback to the user in interactive mode. The real physical
results are still computed by the global illumination algorithms if needed
3.7.4 Photon Shooting
Two-pass methods, photon shooting and image rendering, provide two different places to take
advantage of the temporal coherence among samples. Myszkowski et al. [105] implemented an
algorithm based on caching photons from sampled paths that are subsequently invalidated after a
certain period of time. The lifetime of a photon is derived from perceptual considerations using
the criterion of video and animation quality measurement (AQM). The density estimation particle
tracing algorithm is used to balance the trade-off between maximizing the number of photons
collected in the temporal domain to reduce the stochastic noise and minimizing the time interval in
which the photons were traced to avoid collecting invalid photons. Effectively, the algorithm blurs
each photon in the temporal domain in order to reduce stochastic noise but it also introduces halo
artifacts as a light moving and shadow lingers after an object’s passing. The fundamental problems
43
are that the lifetime of a photon does not depend on the changes in the scene and the contribution
of each photon is not weighted properly across the frames. Dmitriev et al. [32] addressed the
invalidity issue by proposing a selected photon tracing algorithm to identify and update those
invalid photons. The invalid photons are updated by using the periodicity of the Halton random
number sequence to regenerate the sample path for the invalid photon in the current frame. The
algorithm only relieves the usage of invalid photons but it does not actually remove all invalid
photons. In addition the reweighting issue is not addressedby this algorithm. Weber et al. [170]
used space-time bilateral photon filtering to reduce the useof invalid photons by adaptive choice
of filter parameters. However, it does not guarantee conservative results either.
Cammarano et al. [19] solved the motion blur problem in global illumination by proposing a
tempo-spatial photonmapping algorithm. In the first pass, a4D photon map is created by tracing
photons in space and time. The quality propagated is radiantenergy instead of radiant flux. Then,
the relationship between radiance and radiant energy is used to estimate the radiance at each scene
points. The estimated radiance is related to the collected area and the collected period. In short
period of rendering time, the validity of photons should notbe a question. Thus, it works fine
for motion blur rendering. However, when generalized for animation rendering, the validity of
photons becomes an issue. They do not provide proper argument in finding the correct temporal
photons.
Lim [54] separated the global diffuse and specular lightingand used different methods to com-
pute each effect. They used the hierarchical radiosity method to compute the global diffuse effect
and photon tracing to construct caustics textures accounting for the specular effect. They also pro-
vided an algorithm to limit the update of photons in the localmodified regions by a dynamically
modified octree to compactly store the information of particles transiting in the scene. The largest
issue in photon shooting is that the invalidity of copied values introduced bias into the final result.
It is hard to identify invalidity of copied photons.
44
3.7.5 Irradiance Cache
In order to generate high-quality images, the final gathering process needs to be applied for
the image-rendering pass. Tawara et al. [146] separated theirradiance caches into static irradiance
caches which are used to store the irradiance value computedfor all static objects when remov-
ing the dynamic object and dynamic irradiance caches including negative energy are computed
independently for each frame. The static irradiance cachesprovide a temporally consistent illumi-
nation. However, the independent computation of the dynamically cached values may introduce
serious temporal artifacts and thus the quasi Monte Carlo method is used to trace the dynamic
photons and the validity of cached values is an issue in the algorithm. Recently Tawara et al. [145]
extended irradiance caching to the temporal domain by reusing the cached irradiance samples
across frames. The photon maps used for rendering each frameare regenerated because the cost
of generation is relatively low compared to the final gathering process and regeneration can get rid
of the invalid photon problem in the photon shooting process. In every subsequent frame the value
of an irradiance cache is updated by choosing a number of directions to evaluate the irradiance on
this cache. The number of direction is randomly chosen according to a distribution constructed ac-
cording to the age of the irradiance caching. The major issueis still that the updating of irradiance
caches is not conservative. In other words, there may still exist quite a few invalid irradiance caches
which are used to compute the irradiance for the image rendering pass. In addition, the criterion of
choosing which cache to update is heuristic. Smyk et al. [135] introduced “anchor” to keep track
of cache locations to strata where the irradiance is collected. In each frame, the visibility change
of each stratum in each anchor point is checked to do the necessary update of the irradiance in the
anchor point. The updating of irradiance cache is conservative. Thus, it is possible to overcome
invalid caches problem. New anchor points are added if the regions become too sparse and old
anchor points may be removed if no enough valid strata exists. Temporal exploration of irradiance
caches is only helpful for diffuse and moderately glossy surfaces. The algorithm did not handle
the temporal exploration among caustics paths. However, the temporal artifacts caused by caustics
paths in different frames are normally more obvious. They only reuse the cached values in static
objects not in dynamic objects which is perceptually more important. Temporal artifacts may be
45
more serious. Tawara et al. [144] provided a summary of temporal exploration algorithms include
[105, 170, 32, 146, 145, 62] proposed by their groups. They analyzed the pros and cons of each
algorithm and also gave suggestions about the proper applications for each algorithm. In addi-
tion, the photon reuse algorithms separate the direct lighting and indirect lighting. The temporal
coherence in direct lighting is not considered.
3.7.6 Parallel Computation and Accelerate Ray Tracing
A 3D animation scene can be treated as a static one in 4D (space+ time) as proposed by Glass-
ner [51] and Groller et al. [118]. Chapman [22] proposed an object-space intersection algorithm
by transforming the ray through the entire animation into object space and doing the intersection
at once. These algorithms can enhance the speed of ray tracing. However, the accelerations in
intersection are independent from each other and thus, these algorithms do not intend to solve the
issue of temporal artifacts.
With faster hardware and algorithmic improvements, real-time ray tracing is finally within
reach. At a broad level, most of the work in real-time ray tracing algorithms can be classified
into three main categories: improved techniques to computespatial data structures, exploiting ray
coherence and parallel algorithms on shared memory or distributed memory system. The first two
categories are relatively correlated and thus we provide a review of the relative improvement in this
field in the following paragraphs. Then, we describe the advance in parallel computation of ray
tracing. Since ray tracing is the critical component for global illumination, it is obvious that we can
speed up animation rendering by accelerating the efficiencyof ray tracing. There has been a great
deal of work done on accelerating data structures for ray tracing, such as Bounding Volume Hierar-
chies (BVH), Grids, Octrees [50], and Binary Space Partitioning (see e.g. [52, 60] when rendering
statics scene). Gadede and Gunther [45] provided an overview of many spatial subdivisions, along
with the requirement for various application. Recent work in interactive ray tracing, however, has
focused on kd-trees [157, 43, 121, 141] and grids [116, 162] or multilevel grids [110, 120]. When
rendering an animation, we at least need to consider two cost: the cost to update changing scene
46
objects in the acceleration structure and the cost to traverse rays through the acceleration struc-
tures. Instead of tracing one ray at a time, packet tracing creates groups of spatially coherent rays
that are simultaneously traced together through the acceleration structure, where all rays perform
each traversal iteration in lock-step. [21, 121, 141] proposed notably coherent ray tracing in kd-
tree, [162, 55, 5] proposed coherent packet ray tracing in grid-based acceleration structures and
[89, 159] proposed coherent packet ray tracing in BVH-basedacceleration structures. However,
traversal is not the only cost for interactive ray tracing. Spatial data structures which are crucial for
fast ray tracing must be rebuilt or updated as the scene changes, and this can become a bottleneck
for the speed of ray tracing. Thus, [154, 56, 155, 4, 174, 114]focused on rebuilding or updating
a kd-tree-based acceleration structures for dynamic scenes. [72, 71, 86] focused on rebuilding or
updating a grid-based acceleration structure for animatedscenes, and [90, 159, 139, 56, 89, 173]
focused on rebuilding or updating a BVH-based accelerationstructure for animated scenes. [85]
explored the idea of accelerating the operation of intersecting a scene with a ray using constrained
tetrahedralizations. The papers listed here are representative not exclusive and [161] provided an
excellent survey on the topics of improved techniques to compute acceleration structures and ex-
ploiting ray coherence When carefully observing the functionality of these interactive ray tracing
algorithms, they reduce the cost of tracing a ray by using coherent tracing or a better acceleration
structure. They only explore the spatial coherence but not temporal coherence among rays. Thus,
the temporal artifact problem still exists among these algorithms
There exists another research direction to parallel tracing rays in multiple processors to achieve
interactivity in global illumination. [102, 103, 110, 111,134] are examples of parallel ray tracing.
They carefully designed the distribution of ray-tracing tasks to available processors to balance
two factors: load balance and synchronization. The rendering speed can be improved massively.
[119] provided a survey on parallel rendering with ray-tracing, radiosity, and particle tracing. Most
of these research do not intend to explore the temporal coherence among paths except that [94]
proposed to reuse the lens edge by tracking the primary surface points from the previous frame to
start the ray tracing in order to reduce temporal artifacts but the algorithm still does not explored
the coherence of the entire light transport paths. Since PMC-ER algorithm are highly parallel, it
47
can be easily adjusted into these parallel architectures. In Ch. 6 we demonstrate a Condor parallel
rendering system.
3.7.7 Reuse Path Information
Many researchers have explored ways to exploit spatio-temporal coherenc in the image plane to
reduce the computational costs for animation rendering. With a fixed camera, changing materials
or moving objects can be accelerated by storing partial or complete ray trees [17, 101, 75, 130, 10,
41]. However, Raytracing is limited to specular surface only. It has problem in generating diffuse
surface effect.
When the entities allowed to move inside an animated scene are limited, it is easy to reuse light
paths to explore the temporal coherence to reduce temporally-correlated noise. This is relatively
easy to achieve when the viewpoint is the only entity changing inside an animated scene. Since the
viewpoint does not change the illumination inside the scene, the global illumination results at each
surface can be cached in a structure such as a photon map [74] and then the cached value is used
to render the view-dependent frames.
Reprojection has been used in ray tracing to accelerate the generation of animation [6, 76].
These methods save considerable computation by reprojecting primary intersection points from the
previous frame to the current frame and only recomputing therays which are potentially incorrect.
Sbert et al. [125] proposed that if the entity allowed to moveis limited to light sources,
light paths can be reused from the second point of the path in different frames. The weight for
each reconstructed path can be computed using the multiple importance scheme to increase the
rendering efficiency and reduce the temporal artifacts. [124, 123] are extensions of the similar
concept in [125] to randomly shooting radiosity computation. Except the limitation in moving
entities, these algorithms also limit the surface materialof the first vertex from the lights to be
diffuse because a specular surface point will reject the reconnection to light source points because
the relation between input and output direction is a delta function. There exists several methods
that proposed to allow motion only in two entities: the pointlight sources and the viewpoint. The
visibility of the entire scene will not change when the allowable moving entities are point light
48
sources and the viewpoint. When checking the light transport paths, we notice that only the first
link and the last link of the path can possibly change throughthe entire animation. Thus, we
can cache the light paths generated in previous frames and then reweight the contribution to the
current frame based on the multiple-importance framework.Those algorithms achieved significant
improvement in rendering efficiency and reduction in temporal noise. However, the limitation to
point light sources and the movement of the viewpoint and point lights limits the application of
these methods.
Havran et al. [62] reused the sampled paths from bi-directional path tracing to reduce temporal
flickering. The algorithm tries to reuse static paths from the previous frame by checking the visi-
bility of each edge and computing the contribution of the path according to the camera movement.
For each reused path by recomputing the BRDF value at the firsthit point of the camera path is
recomputed. The reweighting schemes for valid paths introduces bias and their algorithm reuses
static paths whose first vertex from the camera has diffuse material.
[95] limited the only moving entity in the animation is the camera. The algorithm relinked the
lens edge and then use an unbiased multiple important techniques to reweight the paths and deposit
the contribution on the image. This saves us the reconstruction of path and inherits the temporal
coherence among paths.
Ghosh et al. [48] applied the framework of Sequential Monte Carlo to the problem of sampling
environment map lighting in an animated sequences. Their work first constructs a set of sample
paths from the environment map light source to the camera in the first frame. For the subsequent
frames they generate a new set of samples by filtering the samples according to the dynamic prod-
uct distribution. Their algorithm provide two advantages:new paths are not constructed from
scratch and the temporal coherence between the current and previous frames is considered. The
former can speed up the rendering and the latter can reduce the temporal flicking. However, their
work is limited to environment map lighting and is not easy toextend to the complete light transport
paths.
49
3.7.8 Decouple the Rendering and Displaying Processes
[16] proposed to directly update the newly available pixel information on currently displayed
buffer instead of background buffer to increase the interactivity by saving the buffer-switch time.
With proper update scheme, the frameless rendering displayexhibits fluid motion. [165, 164]
proposed a mechanism based on point reprojection and progressively adaptive sampling of the il-
lumination to provide fast feedback for interactive use of apixel or ray based renderer. The point
samples are stored in a data structure called Render Cache. [165] used the age of the samples and
other heuristics to identify samples that are likely to be invalid. [165] used temporal weighting
and temporal gradient to evaluate the contribution of irradiance or radiance from different periods
of time. To prevent the holes which can be witnessed with point based approaches, [88, 167]
proposed to use a holodeck structure to keep track of the radiance of rays which have already been
traced inside the entire scene. During rendering, rays inside the visible area are identified by the
holodeck structure and then reprojected back to the image plane, and the server also schedules
necessary update of rays for each beam. [138] maintained a set of textures which represent the
difference between two computations. The corrective textures are updated through an adaptive and
lazy sample tracing. The actual display process draws the approximate interactive solution, aug-
mented by the corrective textures. [133] used tapestry representation which utilizes the unit sphere
centered at the viewpoint as the logical projection surfaceinstead of a planar projection plane. An
icosahedral mesh is created to cover the surface of the sphere. Given a new sample, we perform a
point locating algorithm to identify an existing sphericaltriangle that encloses its projection. Dur-
ing rendering, the spherical triangle is used to reproject the result onto the image plane. This can
relieve the hole problem existing in planar reprojection. [148] proposed reconstruction in object
space by a data structure calledShading Cache. High quality illumination samples are computed
and used to progressively update the shading of patch vertices in a 3D mesh to overcome the gap
possibly existing in point-based methods. The polygonal meshes are rendered through graphic
pipeline. [29] further proposed an adaptively updating pattern on the frame buffer to improve the
visual artifacts caused by temporal aliasing.
50
All these alogrithms proposed to decouple global illumination computations from geometric
projection and image rendering. As such, they are not actualglobal illumination: they do not
aim at computing the illumination, but at providing an interactive display and update of global
illumination solutions, when the actual underlying algorithm cannot provide the required frame
rate. Therefore, they face a similar problem which when the content of images change rapidly, the
system need time to be correctly updated i.e. the intermediate images is not correct.
3.7.9 Hardware Rendering
There are huge amount of research in hardware rendering in recent years. There were a few
research such as [116, 117] which focused on implementing the global illumination algorithm in
graphics hardware architecture for the ability of parallelcomputation. [109] provided a good sur-
vey in this field. However, each frame is still rendered independently and the problem of temporal
artifacts still exists.
Other research in this category emphasizes the interactivity instead of correctness. Approxi-
mation and precomputation are necessary to achieve interactivity. In this section a few interesting
research are reviewed but the list is not thorough. First, reflection is an important global illumina-
tion effect. Diefenbach [31, 30] and Wojdala et al. [171] proposed multipass pipeline rendering to
render mirror reflections for planar surfaces, and [69, 79, 18, 80] proposed methods to handle the
reflection of curved surfaces by using environment mapping.
[1, 2, 106, 112, 3, 64, 13] proposed that all possible radiance on all sample surface points with
all possible view and light directions are precomputed and stored. Then, the precomputed data is
used to render the object under dynamic, local, and low frequency lighting on fly in the rendering
phase. Since the data is precomputed, the rendering result is deterministic and the temporal artifacts
are reduced significantly. However, the precomputation phase still involves the usage of general
global illumination algorithms such photon mapping.
Light map and shadow map methods are proposed by [57, 68, 70, 127, 131] to illuminate and
shadow textured scenes. The use of rasterization hardware allows for interactive display of diffse
51
solution of the scene. However, glossy and specular effectscan only be approximated or must be
added by a separate ray tracing pass
52
Chapter 4
Photorealistic Image Rendering withPopulation Monte Carlo EnergyRedistribution
To generate a physically correct image involves the estimation of a large number of integrals of
path contributions falling on the image plane. It is well known that the integrals have highly cor-
related integrands. However, a standard Monte Carlo rendering algorithm evaluates the integrals
independently. As a result, even a small but important region in the domain is located during the
process. This information is lost to other samples because of the independent sampling. Sample
reuse is an important technique to reduce the variance by exploiting the correlation between in-
tegrals. Markov Chain Monte Carlo algorithms for global illumination, such as Metropolis Light
Transport [152] and Energy Redistribution Path Tracing [25], enable sample reuse by mutating
existing samples into new ones, but the choice of good mutation strategies is non-trivial and has
a major impact on image quality. Population Monte Carlo (PMC) algorithms provide us a tool to
reuse the information collected in previous iterations. PMC Energy Redistribution, adapting the
framework of PMC into the energy redistribution algorithm,exploits information from important
samples through reuse with a mutation process whose mutation strategy is adapted on-the-fly. It is
self-tuning to a large extent.
The PMC Energy Redistribution algorithm iterates on a population of light transport paths.
The population paths are created by tracing the view rays passing through stratified pixel posi-
tions on the image plane by a general Monte Carlo ray tracing algorithm such as path tracing and
53
bidirectional path tracing. In our implementation, we use ageneral path tracing algorithm. The
resampling process eliminates part of the population paths, regenerates new paths to achieve ergoc-
ity, and adapts the kernel functions. The goal of elimination and regeneration is to replace the well
explored or low-contribution paths with new paths generated according to the need of exploring
the image plane evenly for achieving unbiasedness. Furthermore, any information available in the
previous iterations can be used to adapt thekernel functionof each population path that produces
a new population based on the current population. The procedure is then iterated: sample, redis-
tribute, resample, redistribute, resample . . . . The resultis a self-tuning unbiased algorithm which
can explore the important paths locally. Since we use the PMC-ER algorithm to generate static
images, the denotation oft or T is neglected for simplicity in the discussion of this chapter.
4.1 Population Monte Carlo Energy Redistribution
In this section we first present an overview of Population Monte Carlo Energy Redistribution
which can concentrate computation on important light transport paths by exploiting the corre-
lated information among paths and automatically adjust theenergy distributed area for each light
transport path based on information collected in previous iterations. Then, we present the details
including the estimation of average path energy, the kernelfunction which is used to perturb the
current population paths to generate new paths, and the resampling process which eliminates and
regenerates part of the population paths and adaptats the kernel functions according to success rate
of previous perturbations for implementing PMC-ER.
4.1.1 PMC-ER Equal Deposition Algorithm
Figure 4.1 shows the PMC-ER Equal Deposition algorithm. In the preprocess phase, the al-
gorithm first generates a pool of stratified pixel positions.This pool of pixel positions is used to
generate initial population paths and new replacement paths during the resampling process in each
iteration in order to guarantee even exploration of the image plane. Then, the algorithm estimates
the average energy contained in the image,E, and the deposition energy,ed, for each perturbation.
54
The detail of estimation is presented in Section 4.1.2. Initial population paths are created by tracing
rays through unused pixel positions(x, y) from the stratified pool with the path tracing algorithm.
In this work, a path,X, is referred to a light transport path starting from a light,L , scattering
diffusely,D, or specularly,S, inside the scene several times, and ending at the camera,E. The path
is denoted asL(S|D)∗E. Interested readers can refer to [67, 149] for detail. Figure 4.2 and 4.3
shows two examples of such paths.
In each inner loopz, we doNequal perturbations at each path in the population according to
the path’s kernel function,K(s)i (y(z)|Y(z−1)
i ), discussed in Section 4.1.3. After perturbation, the
acceptance probabilityA(Y′(z)i |Y(z−1)
i ) is used to determine whether the path in the population
switches to the new generated pathY′(z)i or stays as the original pathY(z−1)
i before perturbation.
Then,ed energy is deposited on the image plane at the pixel position of the new population path,
Y(z)i .
In the outside loops, the resampling process which is discussed in Section 4.1.4eliminates
well-distributed and low-contribution paths and depositsthe remaining energy of the eliminated
paths, regenerates new paths to explore the image plane evenly, and adapts the kernel function of
each population path. After finishing thes loop, the algorithm deposits the remaining energy of
the population paths onto the image sweep before exiting. Please notice that the remaining energy
in eliminated paths and terminated population paths at the end of the task is deposited to the image
to guarantee the number of Markov Chains starting from each path proportional to its initial energy
for maintaining unbiasedness.
4.1.2 Energy Estimation
When applying energy redistribution with equal depositiondiscussed in Section 3.5.1.2. we
need a criterion to decide whether to start a Markov Chain. The total radiant energy passing
through the image plane,b =∫
Ω L(ydµ(y), is chosen as the criterion in our implementation. Since
b is an integration of radiance carried by all paths, we can useMonte Carlo integration to estimate
it by using the following two equations:
55
1 generate a pool of stratified pixel position
2 estimate theE, ed
3 generate initial population of paths ins = 0
4 for s = 1, · · · , Niteration
5 determineα(s)i for each perturbation
6 for i = 1, · · · , Npopulation
7 if Ei,remain + U(0, 1)E > E
8 for z = 1, · · · , Nequal
9 generateY′(z)i ∼ K
(s)i (y(z)|Y(z−1)
i )
10 Y(z)i = (U(0, 1) < A(Y
′(z)i |Y(z)
i )) ?Y′(z)i : Y
(z−1)i
11 deposited energy onY(z)i
12 Ei,remain− = E
13 w(s)i = Ei,remain
14 resample the population: elimination and regeneration
15 deposit remaining energy in all population paths
Figure 4.1: The PMC-ER Equal Deposition iteration loop.U(0, 1) generates a random numberuniformly distributed between0 and1, andEi,remain is the energy remained in the population path,i, after the inner energy redistribution loops.
E(Y) =L(Y)
pIP (Y)ppath(Y)=
L(Y)AIP
ppath(Y)[4.1]
E =1
N
N∑
i=1
E(Yi) [4.2]
wherepIP is the probability to choose the specific pixel position,ppath is the probability to generate
the path by tracing a view ray through the specific pixel position with path tracing, andE(Y) is
the estimated path energy. Once we choose the starting criterion for equal deposition, we can also
estimate the deposition energy,ed which is:
56
ed =E
Nmutations
[4.3]
whereNmutations = NMC × NMCMC is the expected total number of mutations andNMC
which is the total number of independent path samples andNMCMC = Nequal which is the number
of perturbations in each Markov Chain With this value, the PMC-ER Equal Deposition algorithm
can directly render the final image from the accumulation of energy without the need to calibrate
the total energy of the accumulation image.
4.1.3 The Kernel Function for Each Path
The kernel function for each population path is a conditional kernel, K(s)i (y(z)|Y(z−1)
i ), that
generates a sample pathY′(z)i in iterationz givenY
(z−1)i in iterationz − 1, (see Figure 4.1). we
use a mixture distribution:
K(s)i (y
(z)i |Y(z−1)
i ) =∑
h
α(s)i,hT (y(z)|Y(z−1) : dh) [4.4]
Each component,T (y|Y : d), perturbs an existing path to a new one for local explorationof
the path space according to the perturbing radius,d. Lens and caustics perturbation are two good
candidates for this job. The following is simple description of these two mechanisms:
• Lens perturbation:
Figure 4.2 shows an example of lens perturbation. Lens perturbation is to replace a sub-path
yn−1 · · ·yk of the formEDS∗(L|D). The perturbation method takes the existing path and
moves the image point which it passes. In our case, the new pixel location is uniformly
sampled within a disk of radius, d, a parameter of the kernel component. The remainder
of the path is reconstructed to pass through the new image point and extend the sub-path
through additional specular bounces to be the same length asthe original path. The tenative
transition probability for lens perturbation can be computed as
Td,lens(Y′|Y) =
G(y′n−1,y
′n−2)
Ad
n−k−2∏
j=n−2
G(y′j,y
′j+1)
| cos θj′,in|
57
Figure 4.2: This is a path with the form of LDDSSE and used to demonstrate lens perturbation.We would like to replace the lens sub-pathy5y4y3y2y1 of the form of ESSDD. We first perturbthe pixel position of the original path aty5 by uniformly choosing a point from the perturbing diskand then cast a view ray to pass through the new pixel positionas shown in the bottom to gety′
4.We extend the sub-path through the same specular bounces aty′
4 andy′3 as the correspondingy4
andy3 to gety′2. Then,y′
2 andy1 are linked to form a new lens-perturbed path with the same formof LDDSSE as the original one.
58
Figure 4.3: The is a path with the form of LDSSDE and used to demonstrate caustics pertur-bation. We would like to replace the caustics sub-pathy1y2y3y4y5 of the form DSSDE. At thehead vertex of the caustics sub-path,y1, we perturbed the outgoing light ray direction by(θ, φ),whereθ is uniformly sampled from[0, θmax] andφ is uniformly sampled from[0, 2π], to gety′
2
as showed in the bottom. We extend the sub-path through the same specular bounces aty′2 and
y′3 as the correspondingy2 andy3 to gety′
4. Then,y′4 andy5 are linked to form a new complete
caustics-perturbed path with the same form of LDDSSE as the original one.
59
whereG(y′j,y
′j+1) is the geometric term betweeny′
j andy′j+1, Ad is the area of the per-
turbation, andθj′,in is the angle between the normal of the surface and the direction of the
incoming light ray aty′j.
• Caustics perturbation:
Figure 4.3 shows an example of caustics perturbation. Caustics perturbation is to replace a
caustics sub-path with a suffixym · · ·yk of the form(D|L)S∗D+E. To do this, we generate
a new sub-path starting from the vertexym, the head vertex of the caustics sub-path. The
direction of the segmentym → ym+1 is perturbed by a random amount(θ, φ) and then extend
the sub-path through additional specular bounces to be the same length as the original one.
The (θ, φ) is uniformly sampled from[0, θmax] and [0, 2π] where the central axis,θ = 0,
corresponds to the direction of the original ray andθmax is the range of sampling angle
computed from corresponding perturbation radius,d, by the following equation from [149]:
θmax = θ(d)|yn−1 − yn−2|
∑n−1k=m |yk − yk−1|
[4.5]
whereθ(d) is the angle through which the rayyn → yn−1 needs to be perturbed to change
the image location by a distance ofd pixels. The tentative transition probability for caustics
perturbation can be computed as
Td,caustics(Y′|Y) =
G(ym,ym−1)
2πθmax cos θm,out
m−k−2∏
j=m−1
G(y′j ,y
′j+1)
| cos θj′,out|
whereθj′,out is the angle between the normal of the surface and the direction of the leaving
light ray aty′j .
In original ERPT work, the size of perturbation was a parameter to be set at start-up. In PMC-
ER, we can choose a reasonable set of different sized perturbations in the mixture which is three in
our case. Large perturbation is effective at redistributing information over a wide area, while small
one is benefit for image regions where illumination is changing quickly.
60
When using the kernel function to perturb a path, we first choosed according to the weights,
α(s)i,d , whered is the radius of lens perturbation and
∑
dh′α
(s)i,dh′
= 1. And then either lens or caustics
perturbation is chosen according toγlens = 0.1 andγcaustics = 0.9 in our case which is set to prefer
caustics perturbation when it is possible. We then perturb the current path to generate a new path.
The acceptance probability is calculated accordingly as follow:
A(Y′|Y) = min
1.0,L(Y′)K
(s)i (Y|Y′)
L(Y)K(s)i (Y′|Y)
[4.6]
whereL(Y) is the path contribution. When evaluating the acceptance probability all possible
proposals that might generateY′ from Y must be considered and thus the acceptance probablity
is computed as:
K(s)i (Y′|Y) =
∑
dh′
α(s)i,dh′
(γlensTdh′ ,lens(Y′|Y) + γcausticsTdh′ ,caustics(Y
′|Y))
However, Tierner et al. [147] observed that the acceptance probability can be computed as
A(Y|X) = min
1.0,f(Y)Top−type(X)|Yf(X)Top−type(Y|X)
whereTop−type is the type of perturbation chosen. And the detailed balanceof the perturba-
tion can be still maintained asf(X)Top−type(Y|X) = f(Y)Top−type(X|Y)f(Y)Top−type(X|Y)
f(X)Top−type(Y|X)if
f(Y)Top−type(X|Y) ≥ f(X)Top−type(Y|X); otherwisef(X)Top−type(Y|X)f(Y)Top−type(X|Y)
f(X)Top−type(Y|X)=
f(Y)Top−type(X|Y). Thus, the criterion for unbiasedness and convergence of a Markov Chain is
still achieved. However, Equation?? is chosen to avoid the computation of other possible transition
functions to improve the efficiency of perturbation.
4.1.4 Resampling
The resampling step in this algorithm achieves three purposes: it carries forward to next round
samples that have high energy remaining without flowing out,it provides an opportunity to add
some completely new paths into the population for evenly exploring the image plane, and the
61
information about which perturbations are chosen inside the inner loop guides the adaption of the
kernel functions.
• Elimination :
This step is to eliminate well-explored and low-contribution samples from the population.
When we generate a new population path, the energy of the path, E(Y), is computed using
Equation 4.2 and set it toEremain. At the end of eachs loop, we reduce the energy remaining
in the path byE. We first remove the paths with negative energy. Then, the probability of the
paths surviving in the elimination process is proportionalto the energy remaining in the path,
Eremain. If there is energy remained in the eliminated path, the remained energy is deposited
before we really eliminate the path from the population to guarantee the unbiasedness.
• Regeneration:
Regeneration is to maintain the constant number of paths in the population. It also gives us
the chance to decide where we would like to explore in the nextiterations. For achieving
unbiasedness, we need to evenly explore the image plane. Thus, the regeneration of new
paths is according to the criteria of stratification. In the preprocess phase, we compute
the total stratified number of pixel positions needed for theentire process. Then a pool
of stratified pixel positions is generated according to thatnumber. During the regeneration
process, we keep asking the pool to give us the next unused stratified pixel position. A new
path is generated by tracing through the new pixel position with the path tracing algorithm
and the energy of the path,E(Y), is computed using the Equation 4.2 and set it toEremain.
• Adapt alpha Values
The purpose ofα values is to choose a proper perturbation radius which decides the area of
local exploration according to the success rate of each perturbation. Thus, when a new path
is generated, theα(s)i,h is set to a constant probability for each component, which allows us to
uniformly choose all perturbations. After initialization, acceptance probability of each per-
turbation is tagged with the index of the kernel mixture component and the index of the path.
At the adaptation step, we compute the accumulation of the acceptance probabilities tagged
62
Figure 4.4: The first image on the left is a Cornell Box image computed using PMC-ER EqualDeposition algorithm; the second image is computed using ERPT with 9 SPPs; the third image iscomputed using ERPT with 16 SPPs; and the fourth image is the mutation strategy used duringthe process. The strategy image shows that the kernel function automatically adjusts to perturba-tions with small radii near the physical border and lightingborder to increase the success rate oftransferring image.
with k-th component for each member path and uses it to adjust the mixture probabilities.
We can compute the value ofα(s)i,h as:
α′i,h =
Nperb∑
m=1
Amdh
(Y′(z)i |Y(z−1)
i )δm,h
α(s)i,h = ǫ +
(1− ǫ)α′i,h
∑Nperb
h′=1 α′i,h′
whereNperb is the total number of perturbations applied to this population path, andδm,h = 1
if dh is chosen as the radius of perturbation inm-th perturbation i.em = h
4.2 Results
The perturbation radius is another important factor. It affects the area where the energy can
be distributed to and the success rate of the distribution operation. In the smooth lighting area,
we hope that this radius is large, in order to get a smooth image as soon as possible. However, in
complex lighting areas such as shadow, caustics regions, wehope that it is small or the success
rate of perturbation declines largely. Our algorithm automatically adjusts this aspect through the
process of adaptation ofα values.
63
Figure 4.5: A dragon scene computed using our PMC-ER Equal Deposition atthe top. Thebottom left is the zoom-in of the caustics part computed by PMC-ER equal deposition and thebottom right is the same part computed by ERPT. PMC-ER has fewer artifacts overall. By sharingmore information among paths and by better reusing the high contribution paths, PMC-ER is animprovement over ERPT.
64
Figure 4.6: A room scene computed using our PMC-ER Equal Deposition at the top and ERPTat the bottom. PMC-ER has fewer artifacts overall. By sharing more information among paths andby better reusing the high contribution paths, PMC-ER is an improvement over ERPT.
65
We compared our PMC-ER Equal Deposition algorithm with the energy redistribution path
tracing (ERPT) algorithm on the Cornell Box scene, a dragon scene, and a complex room scene
using the criterion of starting with a similar number of initial PT paths. In all three cases we used a
population size of 5000. There are three perturbation radii: 5, 10, and 50 pixels, respectively. The
range of the perturbed angle for the corresponding causticsperturbation is computed according to
Equation 4.5. In each step inside the inner loop, each membergenerates 16 mutations, and 40% of
the population is eliminated based on its remaining energy and regenerated using the stratification
mechanism. We also use 4 SPPs for estimating the energy contained in an image for both the
PMC-ER and ERPT algorithms.
The Cornell Box image (Figure 4.4) was rendered using our PMC-ER Equal Deposition al-
gorithm with 1000 iterations which roughly has the same total number of initial PT paths as the
image rendered using the ERPT with 8 SPPs. Observing the strategy image, the kernel function
automatically adjusts to small perturbation radii to increase the success rate of energy flowing out
when the exploration is near physical borders and lighting borders such as the shadow and caustics
area and the light edge. We can see that our algorithm removesthe bright spot artifacts from ERPT
algorithm. When we compare our result with an image renderedwith ERPT with 16 SPPs, our
image gets fewer artifacts. In other words PMC-ER renders a more converged image compared
to the corresponding image generated by the ERPT algorithm with the same number of initial PT
paths.
The dragon scene (Figure 4.5) was rendered at 900×900 with 12800 iterations and 20 mutations
for each member in the population inside the loop in comparison with the image rendered using
ERPT with 32 SPPs and 20 mutations to each initial PT path. We can see that the image rendered
with PMC-ER has fewer artifacts than the image rendered withERPT.
The room scene (Figure 4.6) was rendered at 720×405 with 19200 iterations and 20 mutations
for each member in the population inside the loop in comparison with the image rendered using
ERPT with 128 SPPs and 20 mutations to each initial PT path. Wecan see that the image rendered
using PMC-ER has fewer artifacts than the image rendered using ERPT. Note that for all PMC-ER
66
Image Method Time (s) Err Eff
Box1 ERPT(8) 4401.8 0.85 2.7e-4
ERPT(16) 8935.7 0.526 2.1e-4
PMC-ER 5281.2 0.37 5.4e-4
Dragon ERPT(32) 88596.1 1.13 1.0e-5
PMC-ER 97455.7 0.46 2.3e-5
Room ERPT(128) 82656.5 0.052 2.3e-4
PMC-ER 96575.1 0.010 1e-3
Table 4.1: Measurements comparing energy redistribution path tracing (ERPT) with PMC-ER-E,for a roughly equal number of sample rays.
Equal Deposition and ERPT implementations, we did not use the filter proposed in [25] to smooth
the final image.
The statistics for three rendered images is presented in Table 4.1. We use the mean squared
efficiency (Eff ) metric for comparing algorithms, computed as:
Err[
I]
=
∑
pixels err 2
Npixels
, Eff[
I]
=1
T[
I]
×Err[
I]
whereerr is the difference in intensity of a pixel between the rendered value and the ground
truth value,T[
I]
is the running time of the algorithm to render that image andNpixels is the
overall pixel count.Eff[
I]
is a measure of how much longer (or less) you would need to run one
algorithm to reach the quality of another [113]. We can see that our algorithm gets better efficiency
than ERPT algorithm does.
4.3 Discussion
Many PMC kernels in the literature are mixture models. Mixtures are typically formed by
combining several components that are each expected to be useful in some cases but not others.
The adaption step then determines which component are useful for a given input. Mixtures allow
67
otherwise unrelated functions to be combined, such as perturbation with different radii. We would
prefer the kernel function having many components. However, when the kernel function contains
many adaptable parameters, each iteration would requires high adaptive sample counts for gather-
ing proper information to adapt the kernel function. This prevents us from using a larger number of
different perturbing radii. Such a strategy would be appealing for efficiently rendering a scene with
geometries having very different sizes appearing on the image plane, but the adaptive sample count
required to adequately determine the mixture component weights would be too large. Instead we
use three perturbation radii for all images rendered.
We also observe that the deposition energy,ed is important for generating nice results for ERPT
algorithms. Ifed is too small, the algorithm becomes too slow and inefficient but it converges to
smooth results. However, ifed is too large, the algorithm generates bright spots because apath
require higher energy to pass the distribution criterion inorder to start a Markov Chain. Thus,
more paths fail to reach this criterion. In addition, the number of the Markov Chains descreases.
All these increase the chance of accumulating large energy in a close spot. The choice ofed a
trade-off between efficiency and quality.
68
Chapter 5
Efficient Schemes for Monte Carlo MarkovChain Algorithms in Global Illumination
Markov Chain Monte Carlo algorithms such as Metropolis Light Transport (MLT) [152], En-
ergy Redistribution Path Tracing (ERPT) [25], and Population Monte Carlo Energy Redistribution
(PMC-ER) exploit the correlated information among light transport paths to improve the rendering
efficiency. However, MCMC algorithms are limited in achieving higher rendering efficiency due
to the possibly high failure rate in caustics perturbation and the sub-optimal stratified exploration
of the image plane.
The possibly high failure rate in caustics perturbation comes from the bad prediction of the
perturbation angle range in caustics perturbation. The predicted perturbation range depends on
the path and scene properties. If the predicted range is too large, the failure rate of the caustics
perturbation will be high and cause extra high energy to accumulate at some specific spots on
the image plane. As a result, the too large predicted range decreases the rendering efficiency.
In addition to the high failure rate, MCMC algorithms need toimplement the lens and caustics
perturbation methods separately because it is impossible for the original lens perturbation method
to generate a new path from a caustics path with the form ofEDS∗D+(D|L). At the same time,
the calculation of the perturbation angle induces extra cost in caustics perturbation and burden
in predicting the change in the pixel position of a perturbedpath on the image plane for MCMC
algorithms.
69
The ERPT and PMC-ER algorithms carefully generate new pathsaccording to the need of ex-
ploring the image plane evenly for achieving unbiasedness.But this choice is sub-optimal because
some areas on the image plane contain higher perceptual variance than others do. To contribute
more computation effort in reducing the variance in these high perceptual variance regions would
increase the perceptual quality. In addition, some types ofpaths such as caustics paths are visually
important but hard to be found with a general path tracing algorithm. Concentrating more compu-
tation effort on these paths can further improve the rendering efficiency. Thus, evenly exploring the
image plane prevents MCMCs from spending more computation effort in exploring those “high-
perceptual-variance” regions on the image and “hard-to-find-but-important” paths. This limits the
further improvement in rendering efficiency.
To relieve these two limitations, we first introduce a new lens perturbation which can be used
for both lens perturbation and caustics perturbation. Thisnew perturbation method allows us to
control the perturbation by a single and simple lens perturbation radius. The control parameter,
perturbation radius, is intuitive and predictive in the movement of the pixel position of a perturbed
path on the image plane for both lens perturbation and caustics perturbation. It increases the suc-
cess rate of perturbing a caustics path which improves the rendering efficiency in turn. We propose
two methods to generate paths in order to spend more computation effort on exploration of virtually
noisy regions on the image and “hard-to-find-but-important” paths without introducing bias. The
variance-regeneration method generates paths passing through high perceptual variance regions
on the image plane to reduce the variance in this regions for enhancing the perceptual quality of
the rendered image. The caustics-regeneration method generates a set of caustics paths to further
explore the caustics path space. We weight the energy deposited by each perturbation according to
the type of the population path to prevent the new regeneration methods from introducing bias into
the final result. Since we use the PMC-ER algorithm to generate static images, the denotation oft
or T is neglected for simplicity in the discussion of this chapter.
70
5.1 New Schemes to MCMCs
PMC-ER adaptively selects pixels for redistribution, and also adapts algorithm parameters
when tracing paths into the scene. A traced path is then used as the initial state for starting Markov
Chains to redistribute the path’s energy to nearby pixels and find additional light transport paths.
At the same time, PMC-ER adjusts the redistribution area of each path according to the success-
fulness of the previous redistribution. The intuition is that different pixels will find different initial
paths, the information can then be conveyed to neighboring pixels through Markov Chains, and the
size of conveying area will be different from path to path. Inthis section we first briefly discuss
Population Monte Carlo Energy Redistribution with Equal Deposition. We then present the detail
of implementing the new lens mutation and path regenerationmethods.
5.1.1 Population Monte Carlo Energy Redistribution with EqualDeposition
Figure 5.1 shows the PMC-ER Equal Deposition (PMC-ER-E) algorithm. The algorithm first
estimates the average energy contained in the image,E, and the deposition energy,ed, for each
perturbation which are discussed in Chapter 4. The algorithm generates a pool of pixel positions
and a set of caustics paths. The pool of pixel positions and the set of caustics paths are used to
generate initial population paths and replacement paths during the resampling process in each inner
iteration.
In each inner loop,z, we doNequal perturbations at each path in the population according to
the path’s kernel function:
K(s)i (y
(z)i |Y(z−1)
i ) =∑
dh
α(s)i,dh
T (y(z)|Y(z−1) : dh) [5.1]
Then, the acceptance probability can be computed as
A(Y|X) = min
1.0,f(Y)Top−type(X)|Yf(X)Top−type(Y|X)
71
whereTop−type is the type of perturbation chosen. As shown in Ch. 3, the detailed balance can be
maintained under this choice of acceptance probability forachieving the requirement of conver-
gence and unbiasedness.
After perturbation,Ed = R ∗ ed energy is deposited at the pixel position of the new path,
Y(z)i . The constant,R, is an energy-deposited constant which is 1 for the originalPMC-ER-E
algorithm. The constant is adapted according to the properties of the population path discussed in
Section 5.1.3 in order to guarantee the unbiasedness of the algorithm.
In thes loop,E amount of energy is removed fromEi,remain of all paths. Then, the resampling
process eliminates well-distributed and low-contribution paths and redeposits the remained energy
in the eliminated paths, regenerates paths to maintain a constant number of population paths, and
adapts the kernel function of each population path. After finishing thes loop, the algorithm de-
posits the remaining energy of the population paths onto theimage before exiting. Please notice
that the remaining energy in eliminated paths and terminated population paths at the end of the
task is deposited to the image to guarantee the number of Markov Chains starting from each path
proportional to its initial energy for maintaining unbiasedness.
5.1.2 New Lens Perturbation
Details about the mixture kernel funtion are given in Ch. 4. In this section we only focus on
how to use the perturbation method to replace the original caustics perturbation method. Figure 5.2
shows an example of caustics path with the form ofEDS∗(L|D). Original lens perturbation fails
to replace this kind of paths because it is impossible to find exactly the same outgoing direction at
the first specular vertex from the camera when we perturb the pixel position at the camera vertex.
Thus, we need to use caustics perturbation. However, our newlens perturbation method is designed
to perturb a sub-path with the form ofE(D|S)+(SD|S)∗ in order to directly replace the lens and
caustics perturbation methods.
Figure 5.2 shows an example of new lens perturbation for a caustics path. We would like
to perturb the sub-path,xn−1 · · ·xk of the formE(D|S)+(SD|S)∗ by the following steps: First,
the perturbation method moves the pixel position of the existing path on the image plane which
72
1 estimate theE, ed, γ, RG2C
2 generate a pool of pixel positions and a set of caustics paths
3 for s = 1, · · · , Niteration
4 generate initial population of paths ins = 1
5 determineα(s)i for each perturbation
6 for i = 1, · · · , Npopulation
7 if Ei,remain + U(0, 1)E > E
8 for z = 1, · · · , Nmutations
9 generateY′(z)i ∼ K
(z)i (y
(z)i |Y(z−1))
10 Y(z)i = (U(0, 1) < A(Y
′(z)i |Y(z−1)
i ) ?Y′(z)i : Y
(z−1)i
11 depositEd = R ∗ ed on Y(z)i
12 Ei,remain− = E
13 w(o)i = Ei,remain
14 resample the population: elimination and regeneration
15 deposit remaining energy in all population paths
Figure 5.1: The PMC-ER Equal Deposition iteration loop.U(0, 1) generates a random numberuniformly distributed between0 and1. Ei,remain is the energy remaining in the population pathi, after the inner energy redistribution loops.R is the energy-deposited constant discussed inSection 5.1.3.4
it passes. In our case, the new pixel location is uniformly sampled within a disk of radiusd,
a parameter of the kernel component. The disk is centered at the pixel position of the existing
path. The path is reconstructed to pass through the new imagepoint to get the first vertex from
the cameray′n−2. Then, according to the bounce at the vertices in the original path, we use two
different methods to construct the sub-path to have the samelength as the original one. Ifym
wheren−2 ≥ m ≥ k +1 is a specular vertex, we choose a specular bounce to find the next vertex
y′m−1. If ym is a diffuse vertex, we copyym−1 to y′
m−1 and linky′m to y′
m−1 to form the link.
After we reconstruct the sub-path, the tentative transition probability can be computed as
73
Figure 5.2: This is a path with the form of LDDSDE and used to demonstrate the replacement ofcaustics perturbation with the new lens perturbation method. We would like to replace the causticssub-pathy5y4y3y2y1 of the form of EDSD. We first perturb the pixel position of the originalpath aty5 by uniformly choosing a point from the perturbing disk and then cast a view ray to passthrough the new pixel position as showed in the bottom to gety′
4. We link they′4 andy3 to form the
link. Then, we extend the sub-path through the same specularbounce aty′3 as the corresponding
y3 to gety′2. Then,y′
2 andy1 are linked to form a new lens-perturbed path with the same form ofLDDSDE as the original one.
74
Td,lens(Y′|Y) =
G(y′n−1,y
′n−2)
Ad
n−k−2∏
j=n−3
(
G(y′j ,y
′j+1)
| cos θj′,in|: 1?y′
j ⊂ S
)
whereG(y′j ,y
′j+1) is the geometric term betweeny′
j andy′j+1, Ad is the area of the perturba-
tion, andθj′,in is the angle between the normal of the surface and the direction of the incoming
light ray aty′j. This relieves us from the need in the original caustics perturbation method to es-
timate the perturbation angle,θ, for each path. The computation forθ is strenuous and hard to
predict the movement caused by caustics perturbation on theimage plane. When using our new
lens perturbation method, we can use a single pixel perturbation radius to control the movement
of the radius on the image plane. The results show that the control is easier and movement on the
image plane is more predictive.
5.1.3 Resampling
The resampling process consists of three steps:elimination which eliminates well-explored
and low-contribution samples and deposits the remaining energy of the eliminated path into the
image,regenerationwhich maintains the constant number of paths in the population and designs
an exploration pattern in the path space, andadaptation of α valueswhich adjusts the energy
distribution area. In this session, we only focus on the process of regeneration. For the details of
elimination and adaptation please refer to Chapter 4. To generate a new replacement path, we use
three types of regeneration paths: paths passing through a set of stratified pixel positions, paths
passing through a set of pixel positions generated according to the perceptual variance, and a set of
caustics paths tracing from the light sources. To achieve this, we need to have two modifications
in the original algorithm. First, we need to modify step 3 in Figure 4.1 from the generation of a
pool of stratified pixel positions to a pool of pixel positions and a set of caustics paths. Second, we
need to adjustR. The following sections are the implementation details.
75
5.1.3.1 Pixel Positions from Stratification Criterion
Pharr [113] demonstrates how important sampling evenly on the image plane is in reducing
the variance. In energy redistribution, the even distribution of starting pixel positions guarantees
the unbiasedness because every image area has the same probability to generate paths and the
contribution of all paths to the image plane depends on its energy. Thus, we assignNuniform
samples to each pixel.
5.1.3.2 Pixel Positions from Perceptual Variance Criterion
In order to distribute more initial path samples to regions with high perceptual variance, the
algorithm must be able to compute the perceptual-weighted variance of each pixel. Thus, an extra
image varaibleIvariance is added to keep track of the radiance of energy-estimated paths falling
in each pixel during the preprocess of estimating the average path energy. Then, the variance of
sample radiances in each pixel can be computed from the imagevariableIvariance. To account for
perception, the variance in each pixel is divided by the threshold-versus-intensity functiontvi(I)
introduced by Ferweda et al. [42]. A new variable,γj, is introduced to indicate the degree of
requirement for more samples at pixelj. In other words pixels that require further exploration
should have higherγj. Thus,γj should be proportional to the estimate of the perceptually-weighed
variance at each pixel and can be computed as
γ′j =
σ2j
tvi(Ij)
γj =γ′
j∑Npixel
j=1 γ′j
whereIj is the average radiance that falls into pixelj andσ2j is the variance among all sample
radiances falling in pixelj. In order to use variance-regeneration, we first computeγ with the
energy estimation and choose totallyNvariance samples for the entire process. Then each pixel is
assginedNvariance(j) samples according toγj. Then, totallyNuniform + Nvariance(j) samples for
76
pixel j are evenly distributed inside the pixel. All pixel positions from all pixels in the image are
formed the pool of pixel positions.
During the regeneration process, we can ask for a pixel sample from this pool or ask for a new
path from the pool of caustics paths discussed in Section 5.1.3.3. If we get a pixel sample, we
then use the path tracing algorithm to generate a path passing through the new pixel positions. The
unweighted energy of the path is calculated as described in Chapter 4. In Section 5.1.3.4, we will
describe how to weight the deposited energy properly without introducing bias.
In this paper, we set up theNvariance to be large enough for us to use the deterministic sampling
method described in [40] to limit the cost of generating a sample from the variance image,γ’s.
In addition to efficiency, the sample distribution is relatively stratified according to the sampling
probability with deterministic sampling. This implementation can further improve the rendering
efficiency.
5.1.3.3 Caustics Paths
A path tracing algorithm traces paths starting from the camera. However some types of paths
are easier to trace when starting from a light source such as caustics paths. The photonmapping
algorithm uses caustics photons to improve the rendering efficiency. The rendering results in Fig-
ure 5.5 show that caustics paths are hard to find by the path tracing algorithm but they are very
important to generate the smooth caustic regions on the floornear the dragon. Thus, these two
reasons motivate us to use specific types of light paths for the exploration of caustics path space.
In the preprocess we generate a pool ofNcaustics caustics paths in the following way. First, we
choose a light source and then choose a position on that lightsource as the start light vertex. From
the light vertex, we trace a path in the scene as described in [149, 83]. Then, we connect each ver-
tex in the light path to the camera vertex. If the complete path formed is a valid caustics path, we
keep the path in the candidate pool. Finally, we can construct one valid caustics path by randomly
choosing a valid one from the candidate pool. Figure 5.3 shows a example. The criteria for a
caustics path is: first, the length of the path must be over 4 vertices; second, the path must contain
at least one specular vertex; third, the first connection vertex from the camera vertex must be a
77
Figure 5.3: A caustics path is generated by tracing the ray from a light source. Each vertex inthe path is linked to the camera vertex. The algorithm then checks whether the new linked path isa caustics path and if the path is, the algorithm keeps the valid path in the candidate pool. Afterfinishing the whole process, we then randomly choose a path from the candidate pool and put itinto the caustics path pool.
diffuse surface. Without weighting the path energy, these extra “hard-to-find” paths will introduce
bias. In Section 5.1.3.4, we will describe how to weigh the deposited energy without introducing
bias.
5.1.3.4 Weighting the Energy of Newly Regenerated Paths
In original energy redistribution algorithm, we evenly distribute the pixel positions. The energy
distributed ratioR should be one. However, if we unevenly add extra samples intothe image plane
and path space without weighting the energy of each path, theextra sample paths will introduce
biased energy into the image. In this section, we describe how to weight the energy to ensure that
the result is still unbiased.
For the variance-regeneration, each pixel originally hasNuniform samples dropped in the ef-
fective area and this guarantee that the expected energy deposited by paths initialized from each
pixel is statistically the same. To keep the energy deposited from paths initialized in each pixel
statistically equal, we should weight the deposited energyof the path by
78
Nuniform ∗E(j)
NMC
= (Nuniform + Nvariance(j)) ∗Rvariance ∗E(j)
NMC
Rvariance =Nuniform
Nvariance(j) + Nuniform
whereNuniform is the assigned uniform samples per pixel,Nvariance(j) is the number of sam-
ples assigned to pixel(j) by variance-regeneration,E(j) is the expected path energy of a path
generated by tracing through the pixelj and E(j)NMC
is the expected energy deposited by a path
through the pixelj andNMC = Nuniform ∗Npixel. By weighting the deposited energy by a ratio of
Rvariance, we make sure that the total energy expected to be distributed by all paths starting from
that pixel is statistically the same as the one by the originally stratified paths from that pixel.
The situation for caustics-regeneration is different fromthe case of the variance-regeneration.
Caustics paths can be viewed as global paths because they aregenerated by light tracing and
possible to pass through any pixel position on the image plane. Thus, we need to handle them
a little differently. Statistically the ratio of caustics paths and general paths generated from path
tracing algorithm should be fixed. We can use this ratio to weight the energy of all caustics paths in
order to avoid biasedness. The energy-deposited ratio can be calculated by the following equation.
Nexpect ∗Ecaustics
NMC= (Nexpect + Ncaustics) ∗Rcaustics ∗
Ecaustics
NMC
Rcaustics =Nexpect
Ncaustics + Nexpect
whereNexpect is the expected total number of caustics paths existing in the paths generated by
stratified-regeneration,Ncaustics is the total number of caustics paths in the pool of caustics paths,
Ecaustics is the expected path energy of a caustics path, andEcaustics
NMCis the expected deposited
energy of a caustics path. In the preprocess, we estimateRG2C which is the ratio of the total
number of caustics paths to the total number of general paths. Then, we can computeNexpect =
79
RG2C × Npixels × Nuniform for evaluation ofRcaustics. By weighting the energy of each caustics
path by a ratio ofRcaustics, we can guarantee the unbiasedness of the final result.
5.1.3.5 Ratio for Combining These Two Regeneration Methods
In previous section the ratioRcaustics andRvariance are calculated when applying each regen-
eration method separately. However, we would like to combine them to make the system more
efficient. In this section we present how we compute the real ratio R for each population path
generated from different regeneration methods with different path properties. The total expected
energyE delivered to the image plane from the light sources is equal to the sum of the energy
delivered by all paths including the set of caustics paths and the paths generated from the pools of
pixel positions. The paths generated from the pools of pixelpositions can be further splitted into
the pool of caustics paths and non-caustics paths. The totalexpected energyE can be computed as
E =Npath∑
i=1
Ri ×E
NMC
=Npixel−pool∑
i=1
Rpixel−pool ×E
NMC+
Ncaustics∑
i=1
Rcaustics−pool ×E
NMC
=Nnoncaustics−in−pixel−pool
∑
i=1
Rnoncaustics−in−pixel−pool ×E
NMC
+Nexpect∑
i=1
Rexpected−caustics−in−pixel−pool ×E
NMC
+Ncaustics∑
i=1
Rcaustics ×E
NMC
whereRi is the energy-deposited ratio for each path;Npixel−pool is the total number of pixel po-
sitions; Ncaustics is the total number of caustics paths added by the algorithm;When carefully
observing the equation listed above, we can find that the extra caustics paths and the estimated
80
Nexpect caustics paths in the pixel-position pool must be weighted by Rcaustics according to caus-
tics path weighting scheme. SinceNexpect caustics paths come from the pixel pool, they should be
weighted by extraRvariance. Thus, the real ratio,R, of the path deposit energy is separated into the
following three situations: first, if a path is from the pool of pixel positions and is a caustics path,
the ratio should beRvariance(j)×Rcaustics; second, if a path is from the pool of pixel position but is
not a caustics path, the ratio should beRvariance(j); third, if a path is from the pool of caustics path,
the ratio should beRcaustics. By using the ratio accordingly, we can guarantee the unbiasedness.
5.2 Results
In evaluating the performance we compared our implementations with new schemes against
the original PMC-ER Equal Deposition algorithm on a CornellBox (CB) scene, a dragon scene,
and a complex room scene using the criterion of starting withthe same number of initial PT paths.
In all three cases, we used a population size of 5000 and threeperturbation radii: 5, 10, and 50
pixels. In each step inside the inner loop, each member generates 20 perturbations, and 40% of the
population is eliminated based on its remaining energy and regenerated. We use 16 samples per
pixel (SPPs) for estimatingE, ed, γ, andRC2G.
When applying the new schemes to the PMC-ER algorithms, we useNSPP samples per pixels
to compute totallyNtotal initial paths andNiteration iterations, for the PMC-ER algorithms. If
we plug new regenerations into PMC-ER, we will choose totally Nvariance variance-regeneration
pixel positions andNcaustics caustics-regeneration paths for the entire rendering process. Thus,
(Niteration, Nvariance, Ncaustics) are main parameters used to describe the corresponding algorithms
used to render the results in the following sections.
The statistics for three test scenes when using each new scheme separately is presented in
Tables 5.1 and 5.2. Table 5.3 presents the statistics when applying all new schemes with PMC-
ER-E. We used the perceptually-based mean squared efficiency (P − Eff ) metric for comparison.
P − Eff is computed as:
81
Image Method Total IterNiteration Time (s) Err Eff
Box1 E 1225 4769.1 0.0311 6.74e-3
E+Lens* 1225 4683.1 0.0257 8.31e-2
Dragon E 2430 13081.3 3.09 2.47e-5
E+Lens 2430 12640.4 1 7.91e-5
Room E 18800 96575.1 0.0274 3.78e-4
E+Lens 18800 95812.1 0.0208 6.91e-4
Table 5.1: Measurements comparing PMC-ER with original lens and caustics mutation and thestratified regeneration, and PMC-ER with the new lens perturbation method and the stratified re-generation.* +Lens represents that we implement the new lens perturbation method into PMC-ERs.
Err[
I]
=∑
pixels
err 2
tvi(I), P − Eff
[
I]
=1
T[
I]
×Err[
I]
whereerr is the difference in intensity of a pixel between the rendered value and the ground
truth value andT[
I]
is the running time of the algorithm for rendering that image. P − Eff is a
measure of how much longer (or less) you would need to run one algorithm to reach the perceptual
quality of another [113].
5.2.1 Comparison between Old Mutations and New Mutations
Table 5.1 shows the comparison between PMC-ER-E with the original perturbations and PMC-
ER-E with the new lens perturbation method in rendering 3 scenes. Both of them use stratified
regeneration. We would like to identify the improvement introduced by the new lens perturbation.
We use the same total number of iterations for each scene as described in the previous section. The
efficiency is improved by a factor of 1.23 for the CB scene, 3.2for the dragon scene, and 1.82 for
the room scene.
82
Image Method Total IterNiteration Time (s) Err Eff
Box1 E 1225 4769.1 0.0311 6.74e-3
E+Reg* 1225 5366.3 0.0176 1.05e-2
Dragon E 2430 13081.3 3.09 2.47e-5
E+Reg 2430 14296.7 0.985 7.10e-5
Room E 18800 96575.1 0.0274 3.78e-4
E+Reg 18800 98158.9 0.0105 9.70e-4
Table 5.2: Measurements comparing PMC-ER with original lens and caustics mutation with thestratified regeneration, and PMC-ER with original lens and caustics mutation with all regenerationmethods.* +Reg represents implementations of the new regeneration methods in PMC-ERs.
Image Method Total Iter(Niteration) Time (s) Err Eff
Box1 E 1225 4769.1 0.0311 6.74e-3
E+Lens+Reg 1225 5718.4 0.0149 1.17e-2
Dragon E 2430 13081.3 3.09 2.47e-5
E+Lens+Reg 2430 16897.7 0.164 3.61e-4
Room E 18800 96575.1 0.0274 3.78e-4
E+Lens+Reg 18800 107832.5 0.00569 1.62e-3
Table 5.3: Measurements comparing PMC-ER with PMC-ER-E using all the new schemes.
83
The calculation of the perturbation angle proposed in Ch. 4 depends on the path properties and
scene properties. It may predict a too large range of perturbation angle to cause the high failure
rate of caustics perturbation which in turn introduces bright spots into the rendered results. We
observe this effect from rendering the dragon and room scenes. We do an experiment of making
the predicted range 10 times smaller than the predicted value and render the dragon and room
scene. The results are better. Thus, we know that the prediction of the perturbation range is an
important factor to rendering efficiency.
5.2.2 Comparison between Stratified Regeneration and All ThreeRegenerations
Table 5.2 shows the comparison between the original PMC-ER-E algorithm with stratified
regeneration and PMC-ER-E with the new regeneration methods in rendering 3 scenes. Both of
them use the original perturbations. The CB scene is rendered with (Niteration = 1225, Nvariance =
199200, Ncaustics = 108000). The efficiency is improved by a factor of 1.55. The dragon scene
is rendered with(Niteration = 2430, Nvariance = 779700, Ncaustics = 30300). The efficiency is
improved by a factor of 2.87. The room scenes is rendered with(Niteration = 18800, Nvariance =
1363200, Ncaustics = 969600). The efficiency is improved by a factor of 2.56.
5.2.3 Combine Everything into PMC-ER-E
We render the Cornell Box scene (Figure 6.6) at 640×480 with(Niteration = 1225, Nvariance =
199200, Ncaustics = 108000) which is equivalent to 8 SPPs. In comparison, the original PMC-ER
equal deposition algorithm also renders the same scene with(Niteration = 1225). Table 5.3 shows
our algorithm improves the efficiency by a factor of approximate 2.14.
We render the dragon scene (Figure 5.5) at 900×900 with (Niteration = 2430, Nvariance =
779700, Ncaustics = 30300) which is equivalent to 6 SPPs. In comparison original PMC-ER-E
renders the same scene with(Niteration = 2430). Our algorithm shows much better performance
than PMC-ER-E by a factor of approximate 17.53 in Table 5.3.
84
Figure 5.4: The top image is a Cornell Box image computed using PMC-ER-Ealgorithm with all new schemes; the left in the bottom is the cropped image ofthe caustics region for the Cornell Box scene computed usingPMC-ER-E with(Niteration = 1225, Nvariance = 199200, Ncaustics = 108000) plugging in all new schemes;the middle is the cropped image computed by the PMC-ER-E algorithm with (Niteration = 1225);the right is the cropped image computed by the PMC-ER-E algorithm with (Niteration = 2450).
85
Figure 5.5: The top is the rendering result of a dragon scene computed usingPMC-ER-E with all new schemes; the left in the middle row is the cropped im-age of the caustics region below the dragon head computed using PMC-ER-E with(Niteration = 2430, Nvariance = 779700, Ncaustics = 30300) plugging with all new schemes; theright in the middle row is the cropped image computed by the original PMC-ER-E algorithm with(Niteration = 2430); the left in the bottom row is the cropped image computed by the originalPMC-ER-E algorithm with(Niteration = 7290); the right in the bottom row is the cropped imagecomputed by the original PMC-ER-E algorithm in(Niteration = 21870) iterations.
86
Figure 5.6: A room scene computed using PMC-ER-E with(Niteration = 2430, Nvariance = 779700, Ncaustics = 30300) plugging in all new schemes isat the top and the original PMC-ER-E algorithm with(Niteration = 18800) is at the bottom. Thetop result contains more artifacts overall.
87
Our algorithm renders the room scene (Figure 5.6) at 720×405 with(Niteration = 18800, Nvariance =
1363200, Ncaustics = 969600) which is equivalent to 128 SPPs. In comparison original PMC-ER-E
renders the same scene with(Niteration = 18800) Our algorithm shows a better result than PMC-
ER-E by a factor of approximately 4.28 in Table 5.3.
When we check the cropped images of the caustics region underthe glass ball for the CB
scene, the result from our PMC-ER-E algorithm with all new schemes is smoother than the re-
sults from the PMC-ER-E algorithm with equivalent number ofsamples per pixel and with twice
equivalent number of samples per pixel. The improvement is due to several factors. First, variance-
regeneration puts more emphasis on these regions because they contain high perceptual variance.
Second, caustics-regenerations concentrates more computation in the caustics path space i.e. more
exploration is put on the caustics region. When viewing an image, the attention of the viewer is
drawn towards the caustic regions in the image. Thus, improving the quality of the rendered caustic
regions has a large impact on the perception of a rendered image. Finally, the new lens mutation
method increases the perturbation success rate to increasethe exploration of the caustics path for
each population path. As a result, our algorithm can generate smoother caustics regions for the
dragon and CB scene.
Top in Figure 5.6 rendered using PMC-ER-E with all new schemes has fewer artifacts than
bottom in Figure 5.6 rendered using PMC-ER-E. We observe that the variance-regeneration puts
more samples around the regions of the lamp light in the rightof the image. There is no obvious
caustics region in the scene but the bright spots generated during the rendering process mostly
come from the caustics paths. Thus, concentrating more computation in exploration of caustics
path space reduces the variance of the result image. In addition, the failure rate of the caustics
perturbation is high for this scene. With the new lens perturbation method, the success rate can
increase largely. As a result, the rendered image is much smoother.
88
5.3 Discussion
In this section we present the new lens perturbation method and two path regeneration methods.
We demonstrate their effectiveness in the PMC-ER framework. Here we present a short discussion
of how to apply these schemes into the MLT and ERPT frameworks. The new lens perturbation
method can directly replace the lens and caustics perturbation methods. For applying the path
regeneration methods to MLT, we can follow a similar strategy as lens sub-path mutation presented
in [152]. The lens sub-path mutation first generates a set of lens sub-paths and then replace the
lens sub-path in current seed path to generate a new path. To apply the new regeneration methods
to MLT, the variance of the image is first computed with the estimation of average image energy
in the preprocessing phase of MLT, and we can use these regeneration methods to generate a pool
of paths passing through high-variance regions and caustics paths. During the mutation process,
we can replace the current seed path with one of the paths fromthe pool. We can compute the
acceptance probability accordingly and decide whether theseed path transfers to the new generated
path. This should achieve a similar result as presented in our demonstration. When applying it to
the ERPT algorithms, we can estimate the perceptual variance information andRG2C of the scene
with the estimation of the average path energy similar to theone described in Section 5.1.3.2 .
Then, we need to decide how many caustics paths and how many variance-regeneration samples
for the entire process. Once we decide that, we start to generate perceptual paths according to the
variance image computed in the preprocess phase and caustics paths. The energy-deposited ratio
R in Section 5.1.3.4 is computed accordingly to weight the deposited energy of a population path
according to their properties. We should be able to increasethe efficiency of the ERPT algorithms.
There are several important factors including the number ofdifferent perturbation radii, and the
initial α values for each perturbation radius listed in the original PMC-ER algorithm in Chapter 4.
In our algorithm there is another important factor which is the ratio between the total number of
the stratified regeneration paths and the special regeneration paths. If the ratio is high, the image
space exploration rate will be too low. As a result, the variance of those highly explored regions
is reduced significantly but we will have a higher variance inthe other regions. If the ratio is too
89
low, our algorithm reverts to the original PMC-ER algorithm. In the current implementation we
try several values for the ratio and then choose the best values using visual inspection. However,
in the future, we would like to have an algorithm to automatically discover the ratio for rendering
an image with the lowest mean-squared error.
In this section we use the perceptual variance from the radiance of paths as the criterion for the
variance-regeneration. This distributes more samples to high perceptual variance regions according
to the radiance samples but not the real variance in the rendered result. Thus, a new variance
evaluation algorithm based on the rendered results is needed.
We also find that the bright diffuse surfaces near light sources are normally smoother than the
dark diffuse surfaces on the rendered image. This is becausethe criterion to start a MCMC chain is
whether the amount of energy plus a random amount of energy among0 andE is over the average
path energyE . We make a trade-off between speed and variance. Thus, pathspassing through
higher energy regions more easily starts Markov Chains to generate a smoother result. This is
useful for quickly rendering acceptable images but this also causes higher variance on dark regions
which may in turn spend more iterations to gain a smooth result on these dark regions. In the
future, we should separate the rendering process into two phases: to explore all regions evenly and
to explore more on the high energy regions. The even exploration of all regions should use balance
energy transfer algorithm for those paths with energy lowerthan average path energy to make the
low-contribution paths have chance to distribute their energy to smooth the dark regions. After a
fixed number of iterations we should switch to the exploration focusing on high energy regions.
The algorithm used in the second phase should be the same as current PMC-ER. This may result
in a better rendering algorithm.
90
Chapter 6
Physically-based Animation Rendering withMarkov Chain Monte Carlo
A large number of movies employ visual effects using computer graphics. Photorealistic ren-
dering is a critical element to create realistic visual effects. An animation is generally rendered as
a sequence of static images frame-by-frame manner by using Monte Carlo algorithms described
in [113, 39]. As a result the rendered animations normally contain annoying temporal artifacts
such as flickering and shimmering if the computation time is not long enough. Therefore, some
research on global illumination has started to pay attention to enhancing the temporal coherence
among frames in order to remove temporal artifacts. Cachingphotons and irradiances of sample
paths [19, 153, 105, 32, 170, 145, 136, 46] from preceding andfollowing frames can greatly im-
prove the computational efficiency as well as reduce temporal aliasing. [98] caches computed
irradiances from different moments of time on the surfaces.The cached values are used to create
a basis for temporal interpolation of irradiance in order togreatly reduce temporal artifacts. How-
ever, the fundamental problem of these techniques is that the invalidity of cached samples across
frames introduces bias and error into the final result. When only the camera and the point lights
are allowed to move inside an animation scene, it is easy to reuse all light paths by re-evaluating
the path contributions [12, 125, 123, 124, 61] or reweighting samples based on the multiple-
importance framework [95]. However, movement limitationsreduce the utility. Havran et al. [63]
reused the static-object paths from bi-directional path tracing to reduce temporal flickering. For
91
each static candidate path, they first check the validity of the path, i.e. no edge is blocked by mov-
ing objects and then the contribution of valid static paths is evaluated by recomputing the BRDF
value at the first hit point from the camera. The reweighting scheme introduces bias and causes un-
wanted artifacts. In addition, they only intended to reuse static paths not all paths. Ghosh et al. [48]
applied the framework of Sequential Monte Carlo to exploit the temporal coherence among paths
by reweighting the samples in previous frames to generate good samples for the current frame.
However, their work is limited to environment map lighting.In this work we intend to exploit the
temporal coherence among all path integrals in the entire animation whose animated entities are
described by key-framed rigid transformation.
Markov Chain Monte Carlo (MCMC) algorithms such as Metropolis Light Transport (MLT) [152,
81], Energy Redistribution Path Tracing (ERPT) [25], and Population Monte Carlo Energy Redis-
tribution (PMC-ER) in Ch. 4 and 5 have demonstrated the strengths of exploring the spatial coher-
ence among path integrals when rendering a static image. However, since all these algorithms were
originally formulated to render a static image without considering the temporal coherence among
frames in an animation, their results suffer from the temporal flickering artifacts. In this paper we
incorporate MCMC algorithms into exploring the temporal coherence among path integrals.
We formulate the contribution of a light transport path to all frames in an animation as the
sampling distribution for MCMC algorithms. This formulation allows us to extend from rendering
a static image to rendering an animation. As a result, a newtime perturbationmethod is designed
to reuse path samples with similar properties at different moments of time to explore the temporal
coherence among path integrals in order to reduce the temporal artifacts of an animation.
In addition to having an algorithm to explore the temporal coherence, we also need to face
computational limitation when rendering the entire animation sequence. Rendering a high-quality
image for a complex scene normally takes hours to days on a moderate-performance computer. In
this chapter the task of rendering a physically correct and perceptually pleasant image sequence
may take months to finish on a single computer. Our rendering system is designed to render an
animation in parallel. We separate the rendering process into two types of tasks: host and client.
The host task is the main control which creates an input file for each client, collects data from each
92
client, and updates the internal data for next iteration. The client tasks are designed to run on a set of
available computers such as a Condor [26] pool or a computer cluster. Each client task is assigned
a subset of iterative computational tasks. Each computational task contains initial paths traced by
a general path tracing algorithm from a designated frame forenergy redistribution. For efficient
usage of parallel computation, we present a formulation to allow us to locally adjust the rendering
parameters in each task without introducing bias. In this chapter we demonstrate the strength
of exploiting the temporal coherence among paths by building the temporal perturbations on the
PMC-ER algorithms for parallel rendering. The animations rendered with temporal perturbations
are perceptually more pleasant.
6.1 Animation Formulation for Markov Chain MonteCarlo
In Ch. 3 we present a complete derivation of formulation for animation rendering with Markov
Chain Monte Carlo algorithms. Here we presents a short version of the formulation and please
refer to Ch. 3 for more detail. When rendering an animation, the intensity of the j-th pixel of the
k-th frame,Ikj can be expressed as path integrals:
Ikj =
∫
Ωfk
j (xt)dµ(xt) and
fkj (xt) = Le(x
t0,x
t1)G(xt
0,xt1)
Nv−2∏
n′=1
fs(xtn′−1,x
tn′,xt
n′+1)G(xtn′,xt
n′+1)
fs(xtNv−2,x
tNv−1,x
tNv
)G(xtNv−1,x
tNv
)W kj (xt
Nv−1,xtNv
)
= W kj (xt)f(xt)
wheret is the moment when the light transport event occurs,Ω represents the set of all valid light
transport paths that begin on a point of a light surface, interact with the scene surfaces, and end at
the camera for the entire animation denoted asxt, µ(xt) is the surface area product measure for
the pathxt, fkj is the contribution brought by a light transport path to pixel j of frame k, fs is the
bidirectional scattering distribution function,Le is the emittance radiance of a light source,W kj is
93
sensor response,G is the geometry term, andf is the radiance carried by the path. The contribution
function can be decomposed into two components: sensor response,W kj , and the radiance carried
by the path,f . Because the radiance is independent of the response of the given pixel, the radiance
can be reused for all pixels. In other words,b =∫
Ω f(xt)dµ(xt) can be used to express the total
radiant energy delivered through the image sweep as shown inFigure 3.2. The intensity for each
pixel can be estimated by Monte Carlo integration
Ikj =
∫
ΩW k
j (xt)f(xt)dµ(xt) ≈ 1
N
N∑
1
W kj (XTi
i )f(XTi
i )
p(XTi
i )
wherep(XTi
i ) is the importance sampling function used by Monte Carlo integration. The idea of
Markov Chain Monte Carlo algorithms is to generate a set of correlated paths according to their
radiance brought to the image sweep. Through the process, paths mutate, and newly mutated paths
are accepted or rejected with a carefully chosen probability to ensure that paths are sampled ac-
cording to their radiance brought to the image sweep. In other words,p(xt) = f(xt)b
is implicitly
used as the target distribution density. Under this formulation, we can design temporal and spa-
tial mutations by considering the coherence among all validpaths in the entire animation. Since
stratefied exploration of the image plane is important to reduce the variance in rendering images,
the random exploration of the image plane by the Metropolis algorithm is a big limitation in im-
proving rendering efficiency. Thus, a hybrid algorithm, energy redistribution with balance energy
transfer, is purposed to combine the independently sampling path tracing algorithm with the cor-
relatedly sampling Metropolis algorithm. The idea is to generate a set of independent samples by
path tracing as seed paths. The estimation of the pixel intensity can be calculated as
Ikj =
1
NMC
NMC∑
i=1
1
NMCMC
NMCMC∑
l=1
f(XTi
i )
p(XTi
i )W k
j (XTil
il )
[6.1]
Furthermore, if we resample the paths according to its importance energy, the weight of each
perturbation is equal. We can use a similar deterministic sampling strategy in [40] to generate
totally∑
Ei(XTi
i )/E paths whereEi(XTi
i ) =f(X
Tii
)
p(XTii
). In other words, each seed path has a number
of Markov Chains proportional to its energy. The equally weighting scheme now becomes energy
94
redistribution with equal deposition and the estimator canbe calculated as
Ikj =
E
NMC
NMC∑
i=1
Npath∑
m=1
1
NMCMC
NMCMC∑
l=1
W kj (XTil
il )
[6.2]
whereNpath = Nfloor +(0 : 1?E(XTi
i )− (Nfloor +U(0, 1))× E > 0) andNfloor = ⌊E(XTii
)
E⌋. The
variance and expectation analysis are shown in Ch. 3.
6.2 Population Monte Carlo Energy Redistribution
When rendering an animation, we distribute the computationto a task according to the starting
time of initial paths belonging to the same frame. Thus, in the following we will present PMC-
ER Equal Deposition (PMC-ER-E) for a single computational task. We then describe the details
of time perturbation. Finally, we present a formulation to allow us to locally adjust the sampling
parameters for each task without introducing bias.
6.2.1 Population Monte Carlo Energy Redistribution with EqualDeposition
Figure 6.1 shows the PMC-ER-E algorithm executed in a singlecomputational task in a single
computer. At the beginning of each computational task, a pool of pixel positions and a pool of
caustics paths are generated and they are used for the initial population paths and the replacement
paths in the resampling process. In each inner loop,z, we spatially or temporally perturbed each
member pathNequal times to redistribute the energy to the neighborhood of the path according to
a mixture distribution:
K(o)i (Y
(z−1)i → y
(z)i ) =
∑
dh
α(o)i,dh
Tspatial(Y(z−1) : dh → y(z))
+∑
th
α(o)i,th
Ttemporal(Y(z−1) : th → y(z))
When perturbing a path, we first choose one perturbation fromthe set of spatial and temporal
perturbations according to the weights,α(o)i , where
∑
perb α(o)i = 1. The current path is perturbed
to generate a new path according to the perturbation parameterdh for a spatial perturbation andth
95
for a temporal perturbation. The acceptance probability ischosen as
A(Yt1 |Xt0) = min
1.0,f(Yt1)Top−type(X
t1)|Yt1
f(Xt0)Top−type(Yt1|Xt0)
whereTop−type is the type of perturbation chosen. We can observe that the detail balance of this per-
turbation is maintained byf(XT0)Top−type(YT1|XT0) = f(YT1)Top−type(X
T0 |YT1)f(YT1 )Top−type(XT1 |YT1 )
f(XT0 )Top−type(YT1 |XT0 )
if f(YT1)Top−type(XT1|YT1) ≥ f(XT0)Top−type(Y
T1|XT0); otherwisef(XT0)Top−type(YT1|XT0)f(YT1 )Top−type(XT1
f(XT0 )Top−type(YT1
f(YT1)Top−type(XT0|YT1). After each perturbation,Ed = Rk
i ∗ ed energy is deposited at the pixel
position of the newly mutated path,Y(z)i . The energy-deposit constant,Rk
i , is computed according
to the properties of this frame such as the number of samples in each pixel and the number of
caustics paths in order to maintain the energy delivered from each pixel or each type of paths sta-
tistically the same. At the end of eacho loop, E energy is removed from the path’sEi,remain, and
the resampling process is applied. Paths in the population are eliminated according to itsEi,remain.
If there is any energy remaining in the eliminated paths, theremaining energy is deposited by equal
deposition into the image sweep. To maintain the constant number of population, we replace the
eliminated path with newly generated paths from the pool of stratified and variance-regeneration
pixel positions or unused paths from the pool of caustics paths in Ch. 5. We also adaptα values
according to the performance of perturbations. We use the acceptance probability of each pertur-
bation as indication to determine the probability of perturbation choice in Ch. 4. After finishing the
o loop, the algorithm deposits the remaining energy of the population paths onto the image sweep
before exiting. The remaining energy in eliminated paths and terminated population paths at the
end of the task is deposited to the image sweep to guarantee the number of Markov Chains starting
from each path proportional to its initial energy for maintaining unbiasedness.
6.2.2 Time Perturbation
In order to exploit the temporal coherence, we design a new path perturbation strategy called
time perturbation. In this section we first describe how to use our temporal perturbation method
to reconstruct the path temporally and how to calculate the tentative transition probability accord-
ingly.
96
1 getE, ed, T , k, RkC2G, Nk
sample, Nkcaustics, Nequal,γk
Npopulation, Nuniform from the host
2 generate a pool of pixel positions and a set of caustics paths
3 generate initial population of paths ins = 1 at the momentT
4 for o = 1, · · · , Niteration
5 determineα(o)i for each perturbation
6 for i = 1, · · · , Npopulation
7 if Ei,remain + U(0, 1)E > E
8 for z = 1, · · · , Nequal
9 generateY′(z)i ∼ K
(o)i (y(z)|Y(z−1)
i )
10 Y(z)i = (U(0, 1) < a(Y
′(z)i |Y(z)
i )) ?Y′(z)i : Y
(z−1)i
11 depositEd = Rik ∗ ed on Y
(z)i
12 Ei,remain− = E
13 w(o)i = Ei,remain
14 resample the population: elimination and regeneration
15 deposit remaining energy in all population paths
Figure 6.1: The PMC-ER equal deposition iteration loop.E is averaging path energy,ed is depo-sition energy of each perturbation,Rk
C2G is the ratio between caustics and all paths in this frame,Npopulation is the size of the population,Nk
sample is the total number of initial seed paths,Nuniform
is the number of stratified samples per pixel,Nkcaustics is the number of caustics-generation paths,
T is the time when the shutter opens,k is the index of the frame,Nkiteration is the total number of
task iterations,Nkvariance is the number of variance-regeneration paths, andγk
j is the variance-re-generation distribution function. The system use path tracing with a fixed number of samples perpixel to estimateE, ed, Rk
C2G, andγkj which is proportional to the perceptual variance of the sam-
ple radiances in the given pixel frame-by-frame.U(0, 1) generates a random number uniformlydistributed between0 and1. Ei,remain is the energy remaining in the population pathi after theinner energy redistribution loops.Ri
k is the energy-adapted constant discussed in Section 6.2.3.
The main idea behind time perturbation is that when a pathXT0 exists at timeT0, there may be
a correlated pathYT1 whose vertices areobject-basedrigid transformations of the vertices in the
97
pathXT0 [15]. The object-based rigid transformation of a surface point from T0 to T1 is computed
asXT1 = MT1(MT0)−1XT0 whereM t denotes the rigid transformation for the surface from the
object space to the world space att andxt is the world coordinate of the surface pointxobj at t.
A valid perturbed pathYT1 can be generated if the visibility check passes in each edge.However,
this simple time perturbation method may fail if a path contains a specular vertex because
the relation between the input and output direction of a specular vertex is a delta function.
Thus, a more complex scheme is developed for paths containing specular vertices.
Figure 6.2 shows an example of a valid time perturbation of a path with specular vertices.
The step we take to perturb the path is as follow: First, we identify the specular sub-paths of the
form L|[(D|L)D](S+D|S+)+[D(D|E]|E. Second, the specular sub-paths are reconstructed
by eitherbackward samplingor forward sampling. The criterion for choosing a sampling direction
and the details of reconstruction will be described shortly. Finally, un-updated diffuse vertices
are object-based rigid transformed to new locations and un-updated edges are linked and updated
accordingly.
The first and last step are trivial. Thus, we describe the details of step 2. The criterion used to
choose eitherbackward samplingor forward samplingis:
• Case 1: if the path is an eye sub-path of the formL|[(D|L)D](S+D|S+)+E, backward
samplingfrom the eye vertex is chosen to reconstruct the sub-pathto ensure that the eye
sub-path passes through the camera.
• Case 2: if the path is a light sub-path of the formL(S+D|S+)+D(D|E), forward sampling
is chosen to reconstruct the sub-pathto ensure that the light sub-path starts from a valid
light source.
• Case 3: if the path has the form(L|D)D(S+D|S+)+D(D|E), forward samplingandback-
ward samplingare randomly chosen to reconstruct the sub-path.
After we make the choice of the reconstruction direction, wehave to reconstruct the sub-path
in the chosen direction. In the following we provide how to reconstruct a sub-path backward. A
sub-path can be constructed forward in a similar manner except that the direction of transverse
98
is reversed. We would like to reconstruct the sub-path,xt0l · · ·xt0
m of the formD(S+D|S+)+D
by usingbackward sampling. First, the position of vertexm − 1 and m is object-based rigid
transformed fromt0 to t1 to createyt1m andyt1
m−1. Next, the edgeyt1myt1
m−1 is linked to form the
starting ray for constructing the sub-path. Sincext0m−1 is a specular vertex, we choose a specular
bounce atyt1m−1 to find the next vertexyt1
m−2. Then, according to the bounce at the vertices in the
original path, we use two different methods to construct thesub-path to have the same length as the
original one. Ifxt0n wherem− 2 ≥ n ≥ l + 1 is a specular vertex, we choose a specular bounce to
find the next vertex; otherwise we first transform the vertexxt0n−1 to getyt1
n−1 by the object-based
rigid transform method and linkyt1n to yt1
n−1 to form the new edge.
After the path is reconstructed temporally, the tentative transition probability for the time per-
turbation method is computed by considering all possible ways to reconstruct the path. Since we
divide the path into specular sub-paths and diffuse sub-paths, the tentative transition can be com-
puted by multiplying the tentative transition probabilityof each sub-path. For a diffuse sub-path,
the tentative transition probability is one because the transformation and the edge linking is de-
terministic. For a specular sub-path, the tentative transition probability is computed according to
three different cases discussed in the previous paragraph.And the tentative transition probability
of each case must be computed accordingly. The overall tentative transition probability can be
computed as:
Tth,time (Xt0 → Yt1) =1
th
Nsub∏
i=1
T ispecular and
Tspecular =
Tbackward Case1
Tforward Case2
0.5 · Tforward + 0.5 · Tbackward Case3
where Tforward =
∏m−1j=l+1
G(yt1j
,yt1j+1)
| cos θj,out|: 1?yt0
j ⊂ S
Tbackward =∏l+1
j=m−1
G(yt1j
,yt1j−1)
| cos θj,in|: 1?yt1
j ⊂ S
whereyt1j+1, Tforward is the tentative transition probability when reconstructing the sub-path for-
ward,Tbackward is the tentative transition probability when reconstructing the sub-path backward,
99
and θj,in, θj,out is the angle between the normal of the surface and the direction of the incom-
ing/outgoing light ray atyt0j .
6.2.3 Iteratively Distribute Samples in a Frame-by-Frame Man-ner
Since variance-regeneration and caustics-regeneration purposed in Ch. 5 can enhance the ren-
dering efficiency, we would like to have them in our renderingsystem. The weighting scheme is
important to guarantee unbiasedness of the final result at the expense of the computation to for the
distribution of pixel positions and caustics paths in the entire animation. In other words, the system
itself need to generate variance-regeneration pixel positions and caustics-regeneration paths before
creating iterative computation tasks in order to properly weight the energy deposited. However, we
would like to distribute this regeneration to each computation tasks in order to take full advantage
of the parallel rendering. Carefully observing the energy distribution algorithm, we found that the
energy delivered to the entire image sweep can be computed byE =∑NMC
i=1 E (Xi)×NMCMC×ed
where theE (·) is the energy of a seed path. The average path energy can be expressed in a
iteration-frame-based manner asE =∑Niteration
m=1
∑Nframe
k=1
∑Nm,k
sample
n=1 E (Xmkn)×NMCMC × ed
=
∑Niterationm=1
∑Nframe
k=1
E (m, k)
. whereNiteration is the total iterations used,Nframe is the total
number of frames in the animation,Nm,ksample is the total number of samples distributed to thek-th
frame inm-th iteration, andE (m, k) =∑Nm,k
sample
n=1 E (X)×NMCMC × ed denotes the energy deliv-
ered at framek in iterationm. If we can keepE (m, k) statistically unchanged in each frame of
each iteration, the energy delivered to the entire image sweep can keep unchanged. This deriva-
tion allows us to calculate the energy-deposited ratio in each frame according to the number of
assigned stratified pixel position to each pixel, the numberof variance-regeneration pixel positions
in each pixel, and the total number of generated caustics-regeneration paths in that frame without
introducing bias in Ch. 5.
100
6.3 Animation Rendering System
The MCMC formulation allows us to exploit the temporal and spatial coherence among paths
when rendering an animation sequence. However, rendering aphysically-correct animation on a
single computer seems implausible. Thus, in order to make rendering an animation plausible, we
decided to build a parallel animation rendering system. In this section we provide an overview of
our parallel animation rendering system.
6.3.1 Overview
Figure 6.3 shows a flow diagram of our system. Our rendering system divides the process into
two sets of tasks: host and client. The host task maintains the execution of the rendering process.
It is the main controller of our rendering process. Client tasks are iterative computation tasks
distributed to run on a computer pool. Each computational task is to render a specific set of initial
paths. We will discuss the detail of each process in the following paragraphs.
6.3.2 Host Process
The left part in Figure 6.3 shows the processes that are run bythe host. The host, which is the
main control unit in our rendering system, is used to maintain the rendering process in a proper
state: preprocess, script-generation, data-collection,and information-update.
1. Preprocess: this state computes the average path energy,E, deposit energy of each perturba-
tion, ed, the caustics path ratio for each frame,RkG2C , and the degree of sample requirement,
γk.
2. Script-generation: this state creates the input file of each client task according to the infor-
mation collected from the preprocess state or the information gathered in the update state.
The input files are sent to a pool of computers for parallel rendering.
3. Data-collection: when a client process finishes its computation, the intermediate data is
sent to a predefined directory. The host periodically checksthe arrival of data and reads the
101
data from finished client process to update the intermediaterendered results data of the host
process. After collecting data from the client processes, the host turns the control over to the
information-update state.
4. Information-update After collecting all data from all client processed, the host process
computeγk for the next iteration.
6.3.3 Client Process
A client task runs energy redistribution of a subset of initial paths on a computer from a com-
puter pool for parallel rendering. In our system we use the time of initial paths as the distribution
criterion, i.e. a set of initial paths that are valid for a specific framek is distributed to a task. In
Section 6.2.3 we demonstrate that local adjustment of the rendering parameters for each task does
not introduce bias into the final result when a proper weighting scheme is applied.
There are three new functions needed for animation rendering: the first is the description of
object animation; the second is the acceleration structure; the third is the data structure to record
samples. In the following section we give details about these three new functions.
6.3.3.1 Key-framed Rigid Transformation
Currently, we only implement an animated object with a set ofkey-framed rigid transforma-
tions. Each key presents the rigid transformation of a key frame for the object at a moment of time.
Each key containsp, s,q, t to describe the position, scale, orientation, and the moment of time.
We assume that the key frame is dense enough for creating proper interpolation for the object rigid
transformation at any moment with no obvious artifacts. We use linear interpolation (LERP) for
interpolating vector values and spherical linear interpolation (SLERP) for interpolating quaternion
values. The idea is shown in Figure 6.4. With this representation it is easy to compute the rigid
transformation of an object in any moment of time and also to do the rigid transformation of a
vertex needed in Sections 6.2.2.
102
6.3.4 Acceleration Temporal and Spatial Structures in Time
Good acceleration aggregation structure is critical for anefficient global illumination. If an
intersection test needs to query every object in the scene, the compuation for intersection will
be too long for any application. Kd-tree for static rendering improve the efficiency of ray tracing
largely. Thus, we build our animation acceleration structure based on the kd-tree structure concept.
We first can use the same kd-tree structure with a bounding boxstructure to bound the entire
animated in the world through the entire animated period of time. However, this is inefficient
because the animated objects may move in a large extent inside the world, no interesection querry
of the object can be neglected through the acceleration structure. Thus, the keyframed animation
concept inspires us to chop the motion of the entire object motion into Nchop sub-set of motions.
Then, each motion can be bounded by a bounding box tagged withthe start and end time of the
motion. A kdtree is constructed by sorting all bounding boxes from all objects.Nchop is a user-
specified object parameter. If the object moves rapidly,Nchop will be large for avoiding the creation
of a huge bounding box for a motion. The intersection querry is similar to original kd-tree except
that we need to check the time stamp in a ray against the taggedstart and end time of a bounding
box. This is not optimal but it gives us acceptable performance.
6.3.5 Sample Recorder
In static image, we only need to implement a film to record the sample drop in the image plane
by using the spatial filter. However, when we rendering a sequence of images to form an animation,
we need to implement a temporal filter and a data structure to record the samples. In our imple-
mentation, we use a box filter corresponding to the camera shutter. The easiest implementation for
us would be a data structure similar to a scroll of films which contains a set of images and one for
each frame in the animation. However, the memory needed is huge for rendering a long animation.
For example, the memory requirement for rendering 4-secondanimation with a frame rate of 30
frames per second in the resolution of640× 480 is about 1.2 GB. This is probably over the limit
of some machines. In addition, we need to record several auxiliary film-sized information. Thus,
103
this type of implementation is not practical. We have to modified this simple implementation as
follows. Because the initial paths distributed to each iterative computational task based on their
beginning time, the center of energy redistribution is know. Our time perturbation should distribute
most of samples in frames centered arround the beginning frame. Thus, we set up a proper number
of images in the memory space to record the center perturbed samples and then use bucket data
structure in the disk space to record off-centered perturbed samples. Figure 6.5 demonstrates this
concept. When finishing, the images recorded center samplesand the buckets recorded off-center
samples are sent back to a predetermined directory. The collect data daemon will check and read
all data in that directory to update the intermediate results in the host process.
6.4 Results
In our implementation, we attained available computers from a Condor pool [26] to run client
processes in parallel. We used the daemon to communicate with the manager of the Condor pool
to submit the iterative computational tasks, query the completeness of each process in the pool, and
collect data from the pool. Details about the Condor system are available at www.cs.wisc.edu/condor.
To evaluate the performance of exploring temporal coherence among path integrals, we com-
pared animations rendered with time perturbations to animations rendered frame-by-frame on the
Cornell Box (CB) scene, a room scene, a chessboard scene, anda basement scene using the crite-
rion of starting with the same number of initial PT paths and using the same setting for the regener-
ation methods. Since Condor determines the distribution ofcomputational tasks based on the load
of the system and the priority of the user, the completion time of each task is not predictable. It
is not fair to compare the overall rendering time between different rendering methods because the
load of Condor varies from time to time. As a result even the same rendering algorithm may result
in very different overall rendering time. Thus, the statistics listed in table 6.2 are collected from the
computation of the first frame in the first iteration for each animation in a local machine. Table 6.1
also lists the scene-specific parameters. Time needs for rendering an iterative task of an animation
with temporal perturbations requires about 10% more than frame-by-frame. In addition, rendering
104
with temporal perturbations requires roughly additional 60 s to update the intermediate result. In
each task we allocate extra 20 frames around the center frameof the task to record the majority
of perturbed samples to avoid frequently writing off-center sample records to the disk and save
disk space for the off-center records, and this requires extra 200 MB memory. However, when a
perturbed sample falls outside this range, the task writes aoff-center record of time, pixel position,
and radiance into the disk. At the end of the task, the center frames and out-of-core records are all
sent back to the system. The rendering task with temporal perturbations generates additional 48∼54.2 times more data than in the frame-by-frame manner.
In Fig. 6.6, the brightness around the edge of the caustics region rendered in a frame-by-frame
manner changes drastically in consecutive frames. In comparison, the caustics region in the anima-
tion rendered with time perturbations looks smooth and has similar shape and brightness in con-
secutive frames. In accompanied animations, the animationrendered in frame-by-frame manner
has seriously flickering artifacts in the caustics regions.We demonstrate the strength of temporal
exploration to create temporally consistent lighting by intentionally using a less smooth caustics
region. Later in CB2, we use another set of parameters to generate a smoother animation of the
same CB animated scene.
Fig. 6.8 shows the 60-th frame of CB2, room, chess board, and basement scenes rendered
with temporal perturbation. When we check the accompanied animation results, we found that the
temporal artifacts in animations rendered with temporal perturbations are far less than the results
without the temporal perturbations. In addition, the algorithm with temporal perturbations can
generate a converged animation faster than without temporal perturbations.
For the sake of comparison, we rendered a frame in a transientmoment. However, our system
can easily be used to render frames with motion blur by setting a non-zero shutter-open periods
because our time perturbation can easily perturb a path to another path in any moment of time in
the entire animation. Fig. 6.7 demonstrate that motion blurcan be added into a CB animation.
105
Scene Nhostiteration Nuniform Nvariance Scaustics
CB1 1 4 0 0
CB2 1 4 36864000 0
Room 4 4 36864000 1.25
Chess 8 4 36864000 1
Basement 8 4 36864000 1
Table 6.1: Measurements comparing temporal exploration with frame-by-frame. In all cases, weused a population size of 5000, three spatial perturbationshaving radii: 5, 10, and 50 pixels andtwo temporal perturbations having radii: 0.066 and 0.165 s.In the inner loopz, we perturbed eachmember 20 times and eliminated 40% of the population based onits remaining energy and regen-erated new paths to maintain constant number of members in the population. In the preprocess weused 16 samples per pixel (SPPs) for estimatingE, ed, Rk
G2C , andγk. In the rendering process wechoseNhost
iteration as the total number of iterations,Nuniform as the number of sample per pixel forstratified regeneration, andNvariance as the size of the variance-regeneration samples used in aniteration. We computed the number of caustics-regeneration in each frame asScaustics × Nk
expect.Thus, we use(Nhost
iteration, Nuniform, Nvariance, Scaustics) to specify the parameters used to render theanimation scenes.
106
Scene Method Clt(s) Dt(s) Dk (M) Mk (M) Err
CB1 frame-by-frame 2532 1 5 256 1.93e-2
time perturbation 2788 59 240 453 1.82e-2
CB2 frame-by-frame 3213 1 5 266 1.5e-2
time perturbation 3559 62 243 463 6.29e-3
Room frame-by-frame 1743 1 5 606 3.94e-2
time perturbation 1844 65 263 803 8.37e-3
Chess frame-by-frame 2093 1 5 366 5.98e-2
time perturbation 2244 63 253 563 4.51e-1
Basement frame-by-frame 2743 1 5 746 2.61e-1
time perturbation 2954 68 271 943 2.08e-2
Table 6.2: The third column is the time needed to finish the first frame in the first iteration in eachanimation. The fourth column is the time required to update the data from the first frame in the firstiteration. The fifth column is the disk usage for the data whenfinishing the first frame in the firstiteration. The sixth column is the memory consumption in computing the task of the first frame inthe first iteration. The seventh column is the average perceptual error defined in [40] for the entireanimation.
107
6.5 Discussion
Time perturbations find correlated paths across different frames and our system needs to store
the contribution of each correlated path in a set of frames. Theoretically it is possible for our
algorithm to explore all temporal coherence in the entire animation but practically the system has
a limited amount of memory. Therefore, we made a trade-off between the extent of temporal
exploration and the amount of memory and disk usage and the time for updating each process. We
use a set of temporal perturbation radii and the number of perturbations in each Markov Chain
to get the samples distributed roughly in a 20-frame radius from the center of the starting frame
to limit memory usage for center samples and disk space of out-of-center samples. Although
this limits the extent of temporal exploration when the perturbed distance from the starting frame
grows, the failure rate of temporal perturbations grows. This should have slight effect on the rate
of generating temporally consistent animations. A more efficient memory and disk usage and
update scheme or a better task distribution scheme is wantedin the future to avoid the limitation
of temporal exploration radii and to relieve the burden of memory and disk space requirement and
the time for updating task-based data.
Currently we use visual inspection and static perceptual error to check the rendering anima-
tions. This is time consuming and less preferred. There are several animation measurement metrics
developed for video compression. The main issue for video compression is that data loss causes
blockiness artifacts. However, the main issue for MC algorithms is that noise pops up randomly
in the scene and causes serious temporal artifacts. A propermetric must take into account this
independent disturbance. Such a MC-based perceptual metric can help us evaluate the rendering
results and further adjust our kernels to concentrate on perceptually important regions.
In the current implementation we used key-framed rigid transformations to animate entities in
the scene. It is obvious that our algorithm can be easily adjusted to render objects with key-framed
properties such as material, light intensity, and other information. In addition, it is easy to extend
the implementation to include skin-skeleton animation andmorph animation. For skin skeletons
each vertex’s position is related to the skeleton position and orientation at the moment. We can use
108
the intersection point’s parameters(u, v) and three vertices’ positions to compute the position of
corresponding positions for different moments of time. Similarly, the morph animation also uses
(u, v) to compute the corresponding vertex position at different moments. This mechanism allows
us to compute the same corresponding position for each intersection point for different moments
of time.
Our algorithm makes a trade-off between the rendering speedand the image quality by choos-
ing the averaging path energy to determine the number of Markov Chains for each seed path. As
a result, the image quality of the dark regions is generally noisier. When rendering a static image,
the dark regions are perceptually less important and, therefore, the perceptual noise is hard to no-
tice. However, since our perception is sensitive to temporal inconsistency, the noisy dark regions
become perceptually important when rendering an animation. Although temporal exploration can
reduce the variance in dark regions, the variance in dark regions is still perceptible because the
number of Markov Chains distributed to the dark regions is low. Kelemen et al. [81] used multiple
importance framework to combine the independent path samples and correlated path samples to
relieve this problem. This inspires us to think about takingadvantage of the fact that our algorithm
generates a set of independent sample paths before the energy redistribution step. We would like
to develop a hybrid algorithm combining balance transfer and equal deposition. In the algorithm
balance transfer is designed to explore the low-energy paths and equal deposition is designed to
spending more computation on exploring the high-energy paths. This should be able to relieve the
problem of the relatively noisy dark regions. In addition, Kelemen et al. also purposed to map
the creation and mutation of paths to a high-dimension uniform random number cube to increase
the success rate of mutations for enhancing rendering efficiency. Our time and lens perturbation is
designed to use small perturbation on the time and image plane domain to increase the perturbation
success rate. A comparison in the success rate between localperturbations and uniform-cube per-
turbations is needed in the future. We also would like to explore the possibility of combining the
uniform-cube perturbation methods with our adaptation algorithm and spatial and temporal pertur-
bation method by systematically perturbing the random variables used to control the perturbation
of time and the perturbation of lens sub-paths to further increase the success rate of perturbation.
109
Although we demonstrate the strength of temporal exploration based on PMC-ER, the temporal
exploration and local adjustment is easily adapted into theMLT and ERPT frameworks. The new
time perturbation method can be added into the choice of the mutation methods in the ERPT and
MLT algorithms with the implementation of the image sweep. Parallel rendering the animation
with all regeneration methods and locally adjusting parameters is important to get a converged an-
imation quickly. Since the ERPT algorithm has a similar parallel energy-distribution structure, we
can use the same task distribution framework. Each task contains the paths starting from the same
frame. In the preprocess we can estimateγk andRkG2C for each frame withE anded. The compu-
tation of energy deposition ratio discussed in Sec. 6.2.3 can be directly applied. However, applying
parallel local adjustment to MLT is different. We should first generate a seed path per frame and
then a pool of replacement paths consisting of variance-regeneration and caustics-regeneration
paths of the same frame. Then, during the mutation process, we can replace the current seed path
with one of the paths from the pool similar to lens replacement mutation. The acceptance prob-
ability can be computed accordingly to decide whether the exploration path switches to the new
replacement path. This should achieve a similar result as presented in our demonstration.
Another issue is that when we render a sequence of animated frames, the system uses the same
number of iterations for each frame. However, some frames need a larger number of iterations and
some frames need only a few iterations. In this work we manually set up a scene with a reasonable
length and require a similar number of iterations for each frame. However, this is sub-optimal. In
the future we would like to develop an algorithm that can automatically check the convergence of
each frame and terminate the rendering for those frames automatically.
System set up to run roughly between 1000 to 4000 s due to the experiment in Condor system.
We would like to use our algorithm in other kind of computer cluster to have direct control and
direct computation to see the result. There are several possible extensions to our system including
skin skeleton and morph animation to allow us creating complex animations, a better intersection
acceleration structure, and an efficient way to read and write the intermediate images for each
iteration and so on.
110
Figure 6.2: This is a path with the form LDDSSE and is used to demonstrate the backwardsamplingmethod. We would like to transform the pathXt0 at t0 with 6 vertices to a new pathYt1 at t1. Since the original path has the form of case 1, we have to reconstruct the sub-path bybackward sampling. Thus, we first object-based rigid transform the position ofvertices,xt0
5 andxt1
4 to yt15 andyt0
4 and link the edgeyt15 yt1
4 to form the tracing ray at timet1. Then, we extend thesub-path through the same specular bounce atyt1
4 as the correspondingxt04 to getyt1
3 and the samespecular bounce atyt1
3 as the correspondingxt03 to getyt1
2 . Sincext01 andxt0
0 are diffuse surfaces,we only need to object-based rigid transform their positions to getyt1
1 andyt10 and link the edges
of yt12 yt1
1 andyt11 yt1
0 to form a new pathYt1 .
111
Figure 6.3: Overview of the animation rendering system. The left part represents the host processand the right part represents clients. The daemon in the hosthandles the creation of an input filefor each client process, queries the completeness of each client process, and collects the data fromeach client process. A client process receives the input filefrom the host, executes the iterativecomputational task, and sends the results back to the host.
112
Figure 6.4: The keyframed rigit transformation to represent the animation of object.
Figure 6.5: The diagram to represent a movie-like sample recorder.
113
Figure 6.6: The top row of images is coming from the cropped images of the caustics regionof the 27th and 28th frame of the Cornell Box animation rendered in a frame-by-frame manner.The second row of images is coming from the cropped images of the caustics region of the 27thand 28th frame of the Cornell Box animation rendered with ourtime perturbations. Our resultsgenerate a smoother and temporally consistent caustics region in these two consecutive framesand the animation also has a smoother and more consistent caustics region. The caustics regionrendered in frame-by-frame manner is noisy and looks different in shape for consecutive frames.As a result the animation has noisy and twisting caustics regions.
Figure 6.7: These are the cropped image for 5th frames in the Cornell box scene. The left isrendered with a finite shutter open period and the right is rendered with a delta period. We canobserve that the motion blur is added naturally. We can see that the caustics regions, the glass ball,and the shadow regions contain blurred areas around the edge.
114
Figure 6.8: The images from left to right are the 60-th frame generated from CB, room, chessboard, and basement scenes using PMC-ER algorithms with temporal perturbations.
115
Chapter 7
Discussion and Conclusion
The Monte Carlo method is the most general and robust method for solving the global illu-
mination problem. The major challenges in Monte Carlo rendering are to sample the path space
efficiently and to construct good estimators to reduce the variance in the rendered images. This
dissertation has introduced Population Monte Carlo EnergyRedistribution which learns to become
an effective sampler based on the information collected from early iterations for exploration of
the temporal and spatial coherence among paths. This chapter presents a summary of the main
contributions of this work and a discussion of future work.
7.1 Contributions
Sample reuse is an important technique to reduce the sampling variance. Markov Chain Monte
Carlo algorithms such as Metropolis Light Transport [149] and Energy Redistribution Path Trac-
ing [25] apply this technique to sample the path space correlatedly in order to reduce the rendered
variance. However, the choice of good mutation strategies is non-trivial and has a major impact
on image quality. Population Monte Carlo algorithms can automatically adjust their sampling pa-
rameters according to the information collected previously. Thus, we propose to adapt the PMC
framework into the energy redistribution algorithm to become Population Monte Carlo Energy Re-
distribution (PMC-ER). Our algorithm perturbs a population of light transport paths to generate
new paths based on thekernel functionof each path for exploring the spatial and temporal coher-
ence among path integrals. The resampling process eliminates well-explored or low-constribution
116
paths from the population, regenerates new paths to explorethe path space in a designed pattern,
and adapts the kernel function of each path according to the performance of perturbations. The pro-
cedure is then iterated: sample, redistribute, resample, redistribute, resample . . . . The result is a
self-tuning unbiased rendering algorithm being able to explore the spatial and temporal coherence
among light transport paths.
Predicting a proper perturbation range for caustics pertubation is difficult due to the dependence
on the scene and path properties. If the predicted range is too large, the failure rate of the caustics
perturbation will be high to reduce the rendering efficiency. A new lens perturbation method is
proposed to replace both lens perturbation and caustics perturbation with possible increment in
the success rate of perturbation to improve the rendering efficiency. Furthermore, the control
parameter, perturbation radius, is intuitive and predictive in the movement on the image plane for
both lens perturbation and caustics perturbation.
Stratified exploration is important for ERPT and PMC-ER to guarantee unbiasedness. How-
ever, this choice is sub-optimal because the importance of regions on the image are not perceptually
the same i.e. some areas on the image plane contain higher p erceptual variance than others do. In
addition certain types of paths are visually more importantbut hard to find by a general path trac-
ing algorithm. Thus, we propose variance-regeneration to concentrate more computation on high
perceptual variance regions and caustics regeneration to put more effort in exploration of caustics
paths. The deposited energy of each perturbation is weighted according to the path properties and
the way which the path is generated to prevent introducing bias. Both reduce the variance in per-
ceptual important areas with the cost of slight increase in the variance of other areas. The overall
effect is to improve the perception of the rendered image.
Since our perception is sensitive to temperal artifacts. “Frame-by-frame” rendering algorithms
generate artifacts which appear independently among consecutive frames and cause serious tem-
poral artifacts such as flickering and shimmering. In this dissertation we purpose the usage of
PMC-ER to exploits the temporal coherence among path integrals for rendering a general ani-
mated scene. An animated object in the scene is described by key-framed rigid transformation. A
117
new perturbation technique calledtime perturbationis developed to explore the temporal coher-
ence among paths. The time perturbation method is designed to reuse path samples with similar
properties at different moments of time in order to reduce the temporal artifacts of an animation.
Rendering a physically-correct animation for a complex scene may take months for a moderate
computer. In order to generate a converged animation in reasonable time, our system distribute
iterative computational tasks to a pool of computer for parallel rendering. Each computational
task redistributes the energy of a set of initial paths traced through the shutter open period for a
specific frame with locally adjusted parameters. In this dissertation we demonstrate that this local
adaptation of parameters does not introduce bias into the final result. The rendered animations are
perceptually pleasant with much fewer temporal artifacts.
7.2 Future Research
PMC-ER uses sample reuse and adaptation of the kernel function to become a better sampler
for rendering an animation. However, through the entire research we realize that the algorithm
is helpful to improve the rendering efficiency but they are still too slow for practical usage in
industry. Thus, the following is some future works which maybe able to make the algorithm even
more efficient.
In this work we face the difficulty of how to evaluate the perceptual quality of the results.
There are algorithms for evaluating the perceptual qualityof statically rendering images and the
perceptual quality of compressed video. The evaluation in the static images lacks the consideration
of temporal coherence in visual perception. And the evaluation in the compressed video mainly
concentrates on the problem of blockedness which is caused by the loss of information during the
compression process. However, the evaluation in animationrendering does not face the loss of
information but the independent noise in each frame. The temporal inconsistency is very serious to
our perception. We need to consider this effect when evaluating the quality of rendering. Currently
we use visual inspection to check the rendering animations.This is time consuming and hard to
incorporate into the rendering algorithms to make rendering process automatically. Thus a proper
118
metric to evaluate the performance among consecutive frames can allow us to further adjust our
kernel functions to take the entire rendering sequence intoaccount and evaluate the results.
The choice of the ratio between the total number of stratifiedregeneration paths and special
regeneration paths is a trade-off between the degree of stratified exploration of the image plane
and the degree of effort concentrated on exploring perceptually important regions on the image
plane and path space. Currently, the ratio is manually set upat the beginning of the rendering
process. In the future we would like to develop an algorithm to automatically discover the ratio for
rendering an image with the lowest mean-squared error.
In this dissertation variance-generation uses the perceptual variance of sample radiances as the
criterion to distribute pixel samples. The distributions are sub-optimal because they do not reflect
the true need on the rendered images. I would like to have an algorithm to analyze the intermediate
result to allow us to distribute more samples to those noisy regions on the intermediate result.
When observing our rendered results, we find that the bright regions are normally smoother
than the dark regions on a rendered image. The is because we make a trade-off between quality and
speed by choosingE as the criterion to start a Markov Chain. In the future we would like to have
a hybrid algorithm to use balance transfer to explore the low-energy paths and equal deposition to
concentrate more computation on high-energy paths to relieve the problem in the dark regions.
Furthermore, most parts of images are converged quickly except a few areas. Generally, the
noises are caused by a few high-energy paths with high failure rate in perturbations. We think
about further improving the rendering efficiency by: First,a proper criterion is used to identify
these high-failure-high-energy paths and then they are recorded in a list. Second, we analyze the
properties of these difficultly perturbed paths and try to distribute its energy into the intermediate
result. Or we can just remove them if it did not affect the perception of the image.
Although our time perturbation methods can possibly explore the correlation among paths in
the entire animation but requirement of memory to store all perturbed samples will be too high
to fit in a computer system. Therefore, we made a trade-off between the extent of temporal ex-
ploration and the amount of memory usage and the time for updating each process. We set up
reasonable temporal perturbation radii and a proper numberof perturbations to get the samples
119
distributed roughly in a 20-frame radius from the center of the starting frame. However in the
future a better storage or task distribution scheme should be developed to avoid the limitation of
temporal exploration radii.
In addition, temporal and spatial perturbations are implemented separately. The initial number
of perturbations assigned to each will be relatively low. This may cause our algorithm to prefer
a perturbation because of the accidentally high success rate of that perturbation. In the future we
would like to develop a spatio-temporal perturbation algorithm to explore spatial and temporal
domain at the same time to increase the number of initial testperturbations and reduce the chance
of accidentally biasing one perturbation.
In the future skin-skeleton animation and morphing animation seems to be a direct extension
to the current implementation of the key-framed animation.Both skin-skeleton animation and
morphing animation can trace the transformation of a diffuse vertex at different moments of time
by using the intersection’s parameter value(u, v). The ability to trace a diffuse vertex allows us to
generate a correlated paths needed in time perturbation.
In current implementation we assume that the change in lighting condition in the rendered
sequence of frames is low enough to generate converged frames in relatively similar iterations.
However, in general the lighting condition for a long sequence of animation changes largely. Our
algorithm needs to chop up the animated scenes into several small animated scenes for fulfilling
this assumption. In the future we would like to develop an algorithm that can automatically check
the convergence of each frame and terminate the rendering for those frames automatically.
120
LIST OF REFERENCES
[1] Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lightingenvironments, New York, NY, USA, 2002. ACM.
[2] Clustered principal components for precomputed radiance transfer, New York, NY, USA,2003. ACM.
[3] Precomputed local radiance transfer for real-time lighting design, volume 24, New York,NY, USA, 2005. ACM.
[4] Fast kd-tree Construction with an Adaptive Error-Bounded Heuristic. IEEE, Sept. 2006.
[5] Large ray packets for real-time Whitted ray tracing, 2008.
[6] Stephen J. Adelson and Larry F. Hodges. Generating exactray-traced animation frames byreprojection.IEEE Comput. Graph. Appl., 15(3):43–52, 1995.
[7] A.A. Apodaca and L. Gritz.Advanced RenderMan. Morgan Kaufmann, 1999.
[8] James Arvo. Transfer Functions in Global Illumination.In ACM SIGGRAPH ’93 CourseNotes - Global Illumination, pages 1–28. 1993.
[9] Michael Ashikhmin, Simon Premoze, Peter Shirley, and Brian Smits. A variance analysisof the metropolis light transport algorithm.Computer and Graphics, 25(2):287–294, 2004.
[10] Kavita Bala, Julie Dorsey, and Seth Teller. Interactive ray-traced scene editing using raysegment trees. InIn Tenth Eurographics Workshop on Rendering, pages 39–52, 1999.
[11] D. Baum, J. Wallace, M. Cohen, and D. Greenberg. The back-buffer algorithm: an extensionof the radiosity method to dynamic environments. InThe Visual Computer, pages 298–306,1986.
[12] P. Bekaert, Mateu Sbert, and J. Halton. Accelerating path tracing by re-using paths. InRendering Techniques ’02 (Proceedings of the Thirteenth Eurographics Workshop on Ren-dering), pages 125–134, 2002.
121
[13] Aner Ben-Artzi, Ryan Overbeck, and Ravi Ramamoorthi. Real-time brdf editing in complexlighting. ACM Trans. Graph., 25(3):945–954, 2006.
[14] Gonzalo Besuievsky and Xavier Pueyo. Animating radiosity environments through themulti-frame lighting method.Journal of Visualization and Computer Animation, 12:93–106, 2001.
[15] Gonzalo Besuievsky and Mateu Sbert. The multi-frame lighting method: a monte carlobased solution for radiosity in dynamic environments. InProceedings of the eurographicsworkshop on Rendering techniques ’96, pages 185–ff., London, UK, 1996. Springer-Verlag.
[16] Gary Bishop, Henry Fuchs, Leonard Mcmillan, and Ellen J. Scher Zagier. Frameless ren-dering: double buffering considered harmful. pages 175–176, 1994.
[17] N. Briere and P. Pierre. Hierarchical view-dependent structures for interactive scene manip-ulation. InSIGGRAPH ’96, pages 83–90, 1996.
[18] Brian Cabral, Marc Olano, and Philip Nemec. Reflection space image based rendering. InIn SIGGRAPH 99, pages 165–170, 1999.
[19] Mike Cammarano and Henrik Wann Jensen. Time dependent photon mapping. InIn Pro-ceedings of the 13th Eurographics Workshop on Rendering, pages 135–144, 2002.
[20] O. Cappe, A. Guillin, Jean-Michel Marin, and Christian Robert. Population Monte Carlo.Journal of Computational and Graphical Statistics, 13(4):907–929, 2004.
[21] Alan Chalmers and Theresa-Marie Rhyne, editors.Interactive Rendering with CoherentRay Tracing, volume 20. Blackwell Publishers, Oxford, 2001.
[22] J. Chapman, T. W. Calvert, and J. Dill. Spatio-temporalcoherence in ray tracing. InPro-ceedings of Graphics Interface ’91, pages 101–108, Jun 1991.
[23] S.E. Chen. Incremental radiosity: An extension of progressive radiosity to an interactiveimage synthesis system. InSIGGRAPH ’90: Proceedings of the 23th annual conference onComputer graphics and interactive techniques, pages 135–144, 1990.
[24] P.H. Christensen. Industrial-strength global illumination. InSIGGRAPH ’03, Course NotesNo. 27, pages 139–149, 2003.
[25] David Cline, Justin Talbot, and Parris Egbert. Energy redistribution path tracing. InSIG-GRAPH ’05, pages 1186–1195, 2005.
[26] Condor. http:://www.cs.wisc.edu/condor.
[27] Cyrille Damez, Kirill Dmitriev, and Karol Myszkowski.State of the art in global illumi-nation for interactive applications and high-quality animations.Computer Graphics Forum,22(1):55–77, march 2003.
122
[28] Cyrille Damez and Francois Sillion. Space-time hierarchical radiosity. InIn RenderingTechniques ?9, pages 235–246. Springer, 1999.
[29] Abhinav Dayal, Cliff Woolley, Benjamin Watson, and David Luebke. Adaptive framelessrendering. InSIGGRAPH ’05: ACM SIGGRAPH 2005 Courses, page 24, New York, NY,USA, 2005. ACM.
[30] Paul J. Diefenbach.Pipeline rendering: interaction and realism through hardware-basemulti-pass rendering. PhD thesis, Univ. of Pennsylvania, 1996.
[31] Paul J. Diefenbach and Norman I. Badler. Multi-pass pipeline rendering: realism for dy-namic environments. InSI3D ’97: Proceedings of the 1997 symposium on Interactive 3Dgraphics, pages 59–ff., New York, NY, USA, 1997. ACM.
[32] Kirill Dmitriev, Stefan Brabec, Karol Myszkowski, andHans-Peter Seidel. Interactiveglobal illumination using selective photon tracing. InRendering Techniques ’02 (Proceed-ings of the 13th Eurographics Workshop on Rendering), pages 25–36, 2002.
[33] R. Douc, A. Guillin, J. M. Marin, and C. P. Robert. Convergence of adaptive samplingschemes. Technical Report 2005-6, University Paris Dauphine, 2005.
[34] R. Douc, A. Guillin, J. M. Marin, and C. P. Robert. Minimum variance importance samplingvia population Monte Carlo. Technical report, University Paris Dauphine, 2005.
[35] A. Doucet, N. de Freitas, and N. Gordon.Sequential Monte Carlo Methods in Practice.Springer-Verlag, 2001.
[36] George Drettakis and Francois X. Sillion. Interactive update of global illumination usinga line-space hierarchy. InSIGGRAPH ’97: Proceedings of the 24th annual conference onComputer graphics and interactive techniques, pages 57–64. ACM Press/Addison-WesleyPublishing Co., 1997.
[37] Fredo Durand, George Drettakis, and Claude Puech. Thevisibility skeleton: a powerful andefficient multi-purpose global visibility tool. InSIGGRAPH ’97: Proceedings of the 24thannual conference on Computer graphics and interactive techniques, pages 89–100, NewYork, NY, USA, 1997. ACM Press/Addison-Wesley Publishing Co.
[38] Fredo Durand, George Drettakis, and Claude Puech. The3d visibility complex.ACM Trans.Graph., 21(2):176–206, 2002.
[39] Philip Dutre, Kavita Bala Peter Shirley and, and Philippe Bekaert.Advanced Global Illumi-nation. A K Peters Ltd, 2006.
[40] Shaohua Fan.Sequential Monte Carlo Methods for Physically Based Rendering. PhDthesis, University of Wisconsin at Madison, 2006.
123
[41] Sebastian Fernandez, Kavita Bala, Moreno A. Piccolotto, and Donald P. Greenberg. Inter-active direct lighting in dynamic scenes. Technical ReportPCG-00-2, 2000.
[42] James A. Ferwerda, Sumanta N. Pattanaik, Peter Shirley, and Donald P. Greenberg. A modelof visual adaptation for realistic image synthesis. InSIGGRAPH ’96, pages 249–258, 1996.
[43] T. Foley and J. Sugerman. Kd-tree acceleration structures for a gpu raytracer. InProceed-ings of HWWS, pages 15–22, 2005.
[44] D.A. Forsyth, C. Yang, and K. Teo. Efficient radiosity indynamic environments. InPro-ceedings of the Fifth Eurographics Workshop on Rendering, pages 221–231, 1994.
[45] Volker Gaede and Oliver Gunther. Multidimensional access methods.ACM ComputingSurveys, 30:170–231, 1998.
[46] P. Gautron, K. Bouatouch, and S. Pattanaik. Temporal radiance caching. InIEEE Transac-tions on Visualization and Computer Graphics, volume 13, pages 891–901. IEEE ComputerSociety, 2007.
[47] D.W. George, F.X. Sillion, and D.P. Greenberg. Radiosity redistribution for dynamic envi-ronments. InIEEE Computer Graphics and Applications, pages 26–34, 1990.
[48] Abhijeet Ghosh, Arnaud Doucet, and Wolfgang Heidrich.Sequential sampling for dynamicenvironment map illumination. InProc. Eurographics Symposium on Rendering, pages115–126, 2006.
[49] Walter R Gilks, Sylvia Richardson, and David J Spiegelhalter. Markov chain Monte Carloin Practice. Chapman & Hall, 1996.
[50] A. S. Glassner. Space subdivision for fast ray tracing.pages 160–167, 1988.
[51] Andrew S. Glassner. Spacetime ray tracing for animation. IEEE Computer Graphics andApplications, 8(2):60–70, March 1988.
[52] Andrew S. Glassner.An Introduction to Ray Tracing. Academic Press, 1989.
[53] Cindy M. Goral, Kenneth E. Torrance, Donald P. Greenberg, and Bennett Battaile. Mod-eling the interaction of light between diffuse surfaces. InSIGGRAPH ’84: Proceedingsof the 11th Annual Conference on Computer Graphics and Interactive Techniques, pages213–222, 1984.
[54] Xavier Granier and George Drettakis. A final reconstruction framework for an unified globalillumination algorithm.ACM Transactions on Graphics, 23(2), April 2004.
[55] C.P. Gribble, T. Ize, A. Kensler, I. Wald, and S. G. Parke. A coherent grid traversal approachto visualizing particle-based simulation data. pages 758–768, 2006.
124
[56] J. Gunther, H. Friedrich, H. Wald, H.-P. Seidel, and P. Slusallek. Ray tracing animatedscenes using motion decomposition. InProc. Eurographics, 2006.
[57] Paul Haeberli and Kurt Akeley. The accumulation buffer: hardware support for high-qualityrendering. InSIGGRAPH ’90: Proceedings of the 17th annual conference on Computergraphics and interactive techniques, pages 309–318, New York, NY, USA, 1990. ACM.
[58] Hammersley and Handscomb.Monte Carlo Methods. John Wiley & Sons, 1965.
[59] W. K. Hastings. Monte Carlo sampling methods using Markov chains and their applications.Biometrika, 57:97–109, 1970.
[60] Vlastimil Havran. Heuristic Ray Shooting Algorithms. Ph.d. thesis, Department of Com-puter Science and Engineering, Faculty of Electrical Engineering, Czech Technical Univer-sity in Prague, November 2000.
[61] Vlastimil Havran, Jirı Bittner, and Hans-Peter Seidel. Exploiting temporal coherence in raycasted walkthroughs. InSCCG ’03: Proceedings of the 19th spring conference on Computergraphics, pages 149–155, New York, NY, USA, 2003. ACM.
[62] Vlastimil Havran, Cyrille Damez, Karol Myszkowski, and Hans-Peter Seidel. An effi-cient spatio-temporal architecture for animation rendering. In EGRW ’03: Proceedings ofthe 14th Eurographics workshop on Rendering, pages 106–117. Eurographics Association,2003.
[63] Vlastimil Havran, Cyrille Damez, Karol Myszkowski, and Hans-Peter Seidel. An efficientspatio-temporal architecture for animation rendering. InSIGGRAPH ’03: ACM SIGGRAPH2003 Sketches & Applications, pages 1–1, New York, NY, USA, 2003. ACM.
[64] Milos Hasan, Fabio Pellacini, and Kavita Bala. Direct-to-indirect transfer for cinematicrelighting. ACM Trans. Graph., 25(3):1089–1097, 2006.
[65] Milos Hasan, Fabio Pellacini, and Kavita Bala. Matrix row-column sampling for the many-light problem. InSIGGRAPH ’07: ACM SIGGRAPH 2007 papers, page 26, New York, NY,USA, 2007. ACM.
[66] Milos Hasan, Edgar Velazquez-Armendariz, Fabio Pellacini, and Kavita Bala. Tensor clus-tering for rendering many-light animations. InProc. of the 19th Eurographics Symposiumon Rendering. Eurographics Association, 2008.
[67] Paul S. Heckbert. Adaptive radiosity textures for bidirectional ray tracing. InSIGGRAPH’90, pages 145–154, 1990.
[68] T. Heidmann.Real Shadows - Real Time. Iris Universe, 1991.
125
[69] Wolfgang Heidrich and Hans peter Seidel. Realistic, hardware-accelerated shading andlighting. In Proceedings of the 26th annual conference on Computer graphics and interac-tive techniques, pages 171–178, 1999.
[70] Michael Herf and Paul S. Heckbert. Fast soft shadows. InIn Visual Proceedings, SIG-GRAPH 96, page 145, 1996.
[71] T. Ize, P. Shirley, and S.G. Parker. Grid creation strategies for efficient ray tracing. InInteractive Ray Tracing 2006, IEEE Symposium on, pages 27–32, 2006.
[72] T. Ize, I. Wald, C. Robertson, and S.G. Parker. An evaluation of parallel grid constructionfor ray tracing dynamic scenes. InInteractive Ray Tracing 2006, IEEE Symposium on,pages 47–55, 2006.
[73] Henrik Jensen. Global illumination using photon maps.In Rendering Techniques’96: Pro-ceedings of the Eurographics Workshop on Rendering, pages 21–30, 1996.
[74] Henrik W. Jensen. Realistic image synthesis using photon mapping. AK Peters, 2001.
[75] David A. Jevans. Object space temporal coherence for ray tracing. InProceedings of theconference on Graphics interface ’92, pages 176–183, San Francisco, CA, USA, 1992.Morgan Kaufmann Publishers Inc.
[76] Sig Badt Jr. Two algorithms for taking advantage of temporal coherence in ray tracing. InThe Visual Computer, pages 123–132, 1988.
[77] James T. Kajiya. The rendering equation. InSIGGRAPH ’86, pages 143–150, 1986.
[78] Kalos and Whitlock.Monte Carlo Methods, Volume I: Basics. John Wiley & Sons, 1986.
[79] Jan Kautz and Michael D. Mccool. Interactive renderingwith arbitrary brdfs using separableapproximations. InIn Eurographics Rendering Workshop, pages 281–292, 1999.
[80] Jan Kautz, Pere pau Vazquez, Wolfgang Heidrich, Hans peter Seidel, and Aplicacions Udg.A unified approach to prefiltered environment maps. pages 185–196. Springer, 2000.
[81] Csaba Kelemen, Laszlo Szirmay-Kalos, Gyorgy Antal, and Ferenc Csonka. A simple androbust mutation strategy for the metropolis light transport algorithm. volume 21, pages531–540, 2002.
[82] Alexander Keller. Instant radiosity. InSIGGRAPH ’97: Proceedings of the 21st AnnualConference on Computer Graphics and Interactive Techniques, pages 49–56, 1997.
[83] Eric P. Lafortune and Yves D. Willems. Bi-directional path tracing. InProceedings ofCompugraphics, pages 145–153, 1993.
126
[84] Eric P. Lafortune and Yves D. Willems. Bidirectional path tracing. InProceedings ofThird International Conference on Computational Graphicsand Visualization Techniques(Compugraphics ’93), pages 145–153, 1993.
[85] Ares Lagae and Philip Dutre. Accelerating ray tracingusing constrained tetrahedralizations.In SIGGRAPH ’08: ACM SIGGRAPH 2008 posters, pages 1–1, New York, NY, USA, 2008.ACM.
[86] Ares Lagae and Philip Dutre. Compact, fast and robust grids for ray tracing. InSIGGRAPH’08: ACM SIGGRAPH 2008 talks, pages 1–1, New York, NY, USA, 2008. ACM.
[87] Samuli Laine, Hannu Saransaari, Janne Kontkanen, Jaakko Lehtinen, and Timo Aila. Incre-mental instant radiosity for real-time indirect illumination. In Proceedings of EurographicsSymposium on Rendering 2007, pages 277–286. Eurographics Association, 2007.
[88] Greg Ward Larson. The holodeck: A parallel ray-cachingrendering system. InIn Proceed-ings Eurographics Workshop on Parallel Graphics and Visualization, 1998.
[89] Christian Lauterbach, Sung eui Yoon, and Dinesh Manocha. Rt-deform: Interactive raytracing of dynamic scenes using bvhs. InIn Proceedings of the 2006 IEEE Symposium onInteractive Ray Tracing, pages 39–45, 2006.
[90] Jonas Lext and Tomas Akenine-muller. Eurographics 2001 / jonathan c. roberts short pre-sentations towards rapid reconstruction for animated ray tracing. InEurographics, pages311–318, 2001.
[91] Jun S. Liu.Monte Carlo Strategies in Scientific Computing. Spinger-Verlag, 2001.
[92] I. Martin, X. Pueyo, and D. Tosti. A framework for animatio in global illumination. Tech-nical report, Univ.of Girona, 1999.
[93] Ignacio Martın, Xavier Pueyo, and Dani Tost. Frame-to-frame coherent animation withtwo-pass radiosity.IEEE Transactions on Visualization and Computer Graphics, 9(1):70–84, 2003.
[94] W. Martin, W. Martin, S. Parker, S. Parker, E. Reinhard,E. Reinhard, P. Shirley, P. Shirley,W. Thompson, and W. Thompson. Temporally coherent interactive ray tracing.Journal ofGraphics Tools, 2:41–48, 2002.
[95] A. Mendez-Feliu, M. Sbert, and L. Szirmay-kalo. Reusing frames in camera animation.Journal of WSCG, 14(3), 2006.
[96] N. Metropolis and S. Ulam. The Monte Carlo method.Journal of the American StatisticalAssociation, 44:335–341, 1949.
127
[97] Nicholas Metropolis, Arianna W. Rosenbluth, MarshallN. Rosenbluth, Augusta H. Teller,and Edward Teller. Equation of state calculations by fast computing machine.The Journalof Chemical Physics, 21(6):1087–1092, 1953.
[98] Mark Meyer and John Anderson. Statistical acceleration for animated global illumination.ACM Trans. Graph., 25(3):1075–1080, 2006.
[99] S. Muller, W. Kresse, and N. Gatenby. A radiosity approach for the simulation of daylight,1995.
[100] S. Muller and F. Schoeffel. Fast radiosity repropagation for interactive virtual environmentsusing a shadow-form-factor-list, 1994.
[101] K. Murakami and K. Hirota. Incremental ray tracing. InEurographics Workshop on Photo-simulation, Realism and Physics in Computer Graphics, 1989.
[102] M. J. Muuss. Toward real-time ray-tracing of combnatorial solid geometric models. InProceeding of BRL-CAD Symposium, 1995.
[103] M. J. Muuss and M. Lorenzo. High-resolution interactive multispectral missile sensor sim-ulation for atr and dis. InProceeding of BRL-CAD Symposium, 1995.
[104] Karol Myszkowski, Takehiro Tawara, Hiroyuki Akamine, and Hans-Peter Seidel.Perception-informed accelerated rendering of high quality walkthrough sequences. InProc.of the 10th Eurographics Workshop on Rendering, pages 5–18, 1999.
[105] Karol Myszkowski, Takehiro Tawara, Hiroyuki Akamine, and Hans-Peter Seidel.Perception-guided global illumination solution for animation rendering. InSIGGRAPH’01: Proceedings of the 28th annual conference on Computer graphics and interactive tech-niques, pages 221–230. ACM Press, 2001.
[106] Ren Ng, Ravi Ramamoorthi, and Pat Hanrahan. All-frequency shadows using non-linearwavelet lighting approximation.ACM Trans. Graph., 22(3):376–381, 2003.
[107] Jeffry Nimeroff, Julie Dorsey, and Holly Rushmeier. Implementation and analysis of animage-based global illumination framework for animated environments.IEEE Transactionson Visualization and Computer Graphics, 2(4):283–298, 1996.
[108] Rachel Orti, Stphane Riviere, Frdo Durand, and ClaudePuech. Radiosity for dynamicscenes in flatland with.Compute Graphics Forum, 15:237–248, 1996.
[109] John D. Owens, David Luebke, Naga Govindaraju, Mark Harris, Jens Krger, Aaron E.Lefohn, and Timothy J. Purcell. A survey of general-purposecomputation on graphicshardware.Computer Graphics Forum, 26(1):80–113, 2007.
[110] S. Parker, W. Martin, Y. Livnat, P. Sloan, P. Shirley, B. Smits, and C. Hansen. Interactiveray tracing. InSymposium on Interactive 3D Computer Graphics, 1999.
128
[111] S. Parker, W. Martin, Y. Livnat, P. Sloan, P. Shirley, B. Smits, and C. Hansen. Interactiveray tracing for volume visualization.IEEE transaction on COmputer Graphics and Visual-ization, 5(3), 1999.
[112] Fabio Pellacini, Kiril Vidimce, Aaron Lefohn, Alex Mohr, Mark Leone, and John Warren.Lpics: a hybrid hardware-accelerated relighting engine for computer cinematography. InSIGGRAPH ’05: ACM SIGGRAPH 2005 Papers, pages 464–470, New York, NY, USA,2005. ACM.
[113] Matt Pharr and Greg Humphreys.Physically Based Rendering from Theory to Implementa-tion. Morgan Kaufmann, 2004.
[114] Stefan Popov, Johannes Gunther, Hans-Peter Seidel,and Philipp Slusallek. Experienceswith streaming construction of SAH KD-trees. InProceedings of the 2006 IEEE Symposiumon Interactive Ray Tracing, pages 89–94, September 2006.
[115] X. Pueyo, D. Tost, I. Martin, and B. Garcia. Radiosity for dynamic environments. InTheJournal of Visualization and Computer Animation, volume 8, pages 221–231, 1997.
[116] Timothy J. Purcell, Ian Buck, William R. Mark, and Pat Hanrahan. Ray tracing on pro-grammable graphics hardware.ACM Transactions on Graphics, 21(3):703–712, July 2002.ISSN 0730-0301 (Proceedings of ACM SIGGRAPH 2002).
[117] Timothy J. Purcell, Craig Donner, Mike Cammarano, Henrik Wann Jensen, and Pat Hanra-han. Photon mapping on programmable graphics hardware. InProceedings of the ACMSIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware, pages 41–50. Euro-graphics Association, 2003.
[118] Werner Purgathofer, editor.Using Temporal and Spatial Coherence for Accelerating theCalculation of Animation Sequences. North-Holland, September 1991.
[119] E. Reinhard, A. G. Chalmers, and F. W. Jansen. State of the art report overview of parallelphoto-realistic graphics, 1998.
[120] Erik Reinhard, Brian Smits, and Charles Hansen. Dynamic acceleration structures for in-teractive ray tracing. InIn Proceedings of the Eurographics Workshop on Rendering, pages299–306, 2000.
[121] Alexander Reshetov, Alexei Soupikov, and Jim Hurley.Multi-level ray tracing algorithm.In SIGGRAPH ’05: ACM SIGGRAPH 2005 Papers, pages 1176–1185, New York, NY,USA, 2005. ACM.
[122] Christian P. Robert and George Casella.Monte Carlo Statistical Methods. Springer-Verlag,2nd edition, 2004.
129
[123] Mateu Sbert and Francesc Castro. Reuse of paths in finalgathering step with movinglight sources. InInternational Conference on Computational Science 2004, pages 189–196,2004.
[124] Mateu Sbert, Francesc Castro, and John Halton. Reuse of paths in light source animation.In CGI ’04: Proceedings of the Computer Graphics International, pages 532–535, Wash-ington, DC, USA, 2004. IEEE Computer Society.
[125] Mateu Sbert, Laszlo Szecsi, and Laszlo Szirmay-Kalos. Real-time light animation. InComputer Graphics Forum (Proceedings Eurographics 2004), volume 23(3), pages 291–300, 2004.
[126] F. Schoeffel and P.Pomi. Reducing memory requirements for interactive radiosity usingmovement prediction. InEurographics Rendering Workshop, pages 125–134, 1999.
[127] Mark Segal, Carl Korobkin, Rolf van Widenfelt, Jim Foran, and Paul Haeberli. Fast shadowsand lighting effects using texture mapping.SIGGRAPH Comput. Graph., 26(2):249–252,1992.
[128] Benjamin Segovia, Jean-Claude Iehl, Richard Mitanchey, and Bernard Peroche. Bidirec-tional instant radiosity. InProceedings of the 17th Eurographics Workshop on Rendering,to appear, 2006.
[129] Benjamin Segovia, Jean-Claude Iehl, and Bernard Peroche. Metropolis instant radiosity. InProceedings of Eurographics 2007, to appear, 2007.
[130] C. H. Sequin and E. K. Smyrl. Parameterized ray-tracing. InSIGGRAPH ’89: Proceedingsof the 16th annual conference on Computer graphics and interactive techniques, pages 307–314, New York, NY, USA, 1989. ACM.
[131] Jonathan Shade, Dani Lischinski, David H. Salesin, Tony DeRose, and John Snyder. Hi-erarchical image caching for accelerated walkthroughs of complex environments. InSIG-GRAPH ’96: Proceedings of the 23rd annual conference on Computer graphics and inter-active techniques, pages 75–82, New York, NY, USA, 1996. ACM.
[132] E. Shaw. Hierarchical radiosity for dynamic environments. InComputer Graphics Forum(Proceedings Eurographics 1997), volume 16, pages 107–118, 1997.
[133] Maryann Simmons and Carlo H. Sequin. Tapestry: A dynamic mesh-based display repre-sentation for interactive rendering. InProceedings of the 11th Eurographics Workshop onRendering, pages 329–340, 2000.
[134] S.J.Gortler and K.Myszkowski, editors.Interactive Distributed Ray Tracing of Highly Com-plex Models. Springer, 2001. available at http://graphics.cs.uni-sb.de/ wald/Publications.
130
[135] Miloslaw Smky, Shin-ichi Kinuwaki, Roman Durikovic,and Karol Myszkowski. Tempo-rally coherent irradiance caching for high quality animation rendering. InThe EuropeanAssociation for Computer Graphics 26th Annual Conference EUROGRAPHICS 2005, vol-ume 24 ofComputer Graphics Forum, pages xx–xx, Dublin, Ireland, 2005. Blackwell.
[136] M. SMyk, S. Kinuwaki, R. Durikovic, and K. Myszkowski.Temporally coherence irradi-ance caching for high quality animiation rendering. InComputer Graphics Forum (Pro-ceedings Eurographics 2005), 2005.
[137] Jerome Spanier and Ely M. Gelbard.Monte Carlo principles and neutron transport prob-lems. Reading, Mass., Addison-Wesley Pub. Co, 1969.
[138] Marc Stamminger, Jorg Haber, Hartmut Schirmacher, and Hans peter Seidel. Walkthroughswith corrective texturing. Intal Signal Processing. California Technical Publishing, pages377–388, 2000.
[139] Gordon Stoll, William R. Mark, Peter Djeu, Rui Wang, and Ikrima Elhassan. Razor: Anarchitecture for dynamic multiresolution ray tracing. Technical report, University of Texasat Austin, 2005.
[140] K. Sung, A. Pearce, and C. Wang. Spatial-Temporal Antialiasing. IEEE Transactions onVisualization and Computer Graphics, 8(2):144–153, 2002.
[141] Jorg Schmittler Sven Woop and Philipp Slusallek. Rpu: A programmable ray processingunit for realtime ray tracing. InProceedings of ACM SIGGRAPH 2005, pages 434–444,July 2005.
[142] Laszlo Szirmay-Kalos. Monte Carlo methods for global illumination. InSpring Conferenceof Computer Graphics?9, pages 1–28, 1999. Invited talk.
[143] Eric Tabellion and Arnauld Lamorlette. An approximate global illumination system forcomputer generated films. InSIGGRAPH ’04: Proceedings of the 31st Annual Conferenceon Computer Graphics and Interactive Techniques, pages 469–476, 2004.
[144] T. Tawara, K. Myszkowski, K. Dmitriev, V. Havran, C. Damez, and H.-P. Seidel. Exploitingtemporal coherence in global illumination. InSCCG ’04: Proceedings of the 20th springconference on Computer graphics, pages 23–33. ACM Press, 2004. Invited Talk.
[145] T. Tawara, K. Myszkowski, and H.-P. Seidel. Exploiting temporal coherence in final gather-ing for dynamic scenes. InCGI ’04: Proceedings of the Computer Graphics International(CGI’04), pages 110–119. IEEE Computer Society, 2004.
[146] Takehiro Tawara, Karol Myszkowski, and Hans-Peter Seidel. Localizing the final gatheringfor dynamic scenes using the photon map. In Gunther Greiner, Heinrich Niemann, ThomasErtl, Bernd Girod, and Hans-Peter Seidel, editors,Proceedings of Vision, Modeling, andVisualization VMV 2002, pages 69–76, Erlangen, Germany, November 2002. AkademischeVerlagsgesellschaft Aka.
131
[147] Luke Tierney. A note on Metropolis-Hastings kernels for general state spaces.The Annalsof Applied Probability, 8(1):1–9, 1998.
[148] Parag Tole, Fabio Pellacini, Bruce Walter, and DonaldP. Greenberg. Interactive globalillumination in dynamic scenes. InSIGGRAPH ’02: Proceedings of the 29th Annual Con-ference on Computer Graphics and Interactive Techniques, pages 537–546. ACM Press,2002.
[149] Eric Veach. Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis,Stanford University, 1997.
[150] Eric Veach and Leonidas J. Guibas. Bidirectional estimators for light transport. InProc. ofthe 5th Eurographics Workshop on Rendering, pages 147–162. Eurographics Association,1994.
[151] Eric Veach and Leonidas J. Guibas. Optimally combining sampling techniques for MonteCarlo rendering. InSIGGRAPH ’95: Proceedings of the 22nd Annual Conference on Com-puter Graphics and Interactive Techniques, pages 419–428, 1995.
[152] Eric Veach and Leonidas J. Guibas. Metropolis light transport. InSIGGRAPH ’97, pages65–76, 1997.
[153] V. Volevich, K. Myszkowski, A. Khodulev, and E. Kopylov. Using the visual differencespredictor to improve performance of progressive global illumination computations. vol-ume 19, pages 122–161, New York, NY, USA, 2000. ACM.
[154] I. Wald, C. Benthin, and P. Slusallek. Distributed Interactive Ray Tracing of DynamicScenes. InProc. of the IEEE Symposium on Parallel and Large-Data Visualization andGraphics, 2003.
[155] I. Wald and V. Havran. On building good kd-trees for raytracing, and on doing this in o(nlog n). Technical report, Univ.of Utah, 2006.
[156] I. Wald, T. Kollig, C. Benthin, A. Keller, and P. Slusallek. Interactive Global Illumination.In Proc. of the 13th Eurographics Workshop on Rendering, pages 15–24, 2002.
[157] Ingo Wald. Realtime Ray Tracing and Interactive Global Illumination. PhD thesis, Com-puter Graphics Group, Saarland University, 2004.
[158] Ingo Wald, Carsten Benthin, and Philipp Slusallek. Interactive global illumination in com-plex and highly occluded environments. InEGRW ’03: Proceedings of the 14th Eurograph-ics workshop on Rendering, pages 74–81, Aire-la-Ville, Switzerland, Switzerland, 2003.Eurographics Association.
[159] Ingo Wald, Solomon Boulos, and Peter Shirley. Ray tracing deformable scenes using dy-namic bounding volume hierarchies.ACM Transactions on Graphics, 26:2007, 2007.
132
[160] Ingo Wald, Thomas Kollig, Carsten Benthin, AlexanderKeller, and Philipp Slusallek. In-teractive global illumination using fast ray tracing. InEGRW ’02: Proceedings of the 13thEurographics workshop on Rendering, pages 15–24, Aire-la-Ville, Switzerland, Switzer-land, 2002. Eurographics Association.
[161] Ingo Wald, William R. Mark, Johannes Gunther, Solomon Boulos, Thiago Ize, Warren Hunt,Steven G. Parker, and Peter Shirley. State of the art in ray tracing animated scenes. In DieterSchmalstieg and Jirı Bittner, editors,STAR Proceedings of Eurographics 2007, pages 89–116. The Eurographics Association, September 2007.
[162] Ingo Wald, Ize Thiago, Andrew Kensler, Aaron Knoll, and Steven G. Parker. Ray TracingAnimated Scenes Using Coherence Grid Traversal. InSIGGRAPH ’06: Proceedings of the32th annual conference on Computer graphics and interactive techniques, 2005.
[163] Bruce Walter, Adam Arbree, Kavita Bala, and Donald P. Greenberg. Multidimensionallightcuts. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Papers, pages 1081–1088, NewYork, NY, USA, 2006. ACM.
[164] Bruce Walter, George Drettakis, and Donald Greenberg. Enhancing and optimizing the ren-der cache. In Paul Debevec and Simon Gibson, editors,Thirteenth Eurographics Workshopon Rendering (2002), page June. Eurographics, ACM Press, 2002.
[165] Bruce Walter, George Drettakis, and Steven Parker. Interactive rendering using the rendercache. In D. Lischinski and G.W. Larson, editors,Rendering techniques ’99 (Proceedingsof the 10th Eurographics Workshop on Rendering), volume 10, pages 235–246, New York,NY, Jun 1999. Springer-Verlag/Wien.
[166] Bruce Walter, Sebastian Fernandez, Adam Arbree, Kavita Bala, Michael Donikian, andDonald P. Greenberg. Lightcuts: a scalable approach to illumination. ACM Trans. Graph.,24(3):1098–1107, 2005.
[167] Gregory Ward and Maryann Simmons. The holodeck ray cache: an interactive renderingsystem for global illumination in nondiffuse environments. ACM Trans. Graph., 18(4):361–368, 1999.
[168] Gregory J. Ward and Paul Heckbert. Irradiance gradients. In Proceedings of the 3rd Euro-graphics Workshop on Rendering, pages 85–98, 1992.
[169] Gregory J. Ward, Francis M. Rubinstein, and Robert D. Clear. A ray tracing solution fordiffuse interreflection. InSIGGRAPH ’88: Proceedings of the 15th annual conference onComputer graphics and interactive techniques, pages 85–92, 1988.
[170] Markus Weber, Marco Milch, Karol Myszkowski, Kirill Dmitriev, Przemyslaw Rokita, andHans-Peter Seidel. Spatio-temporal photon density estimation using bilateral filtering. InCGI ’04: Proceedings of the Computer Graphics International (CGI’04), pages 120–127.IEEE Computer Society, 2004.
133
[171] A. Wojdala, M. Gruszewski, and K. Dudkiewicz. Using harware texture mapping for ef-ficient image synthesis and walkthrough with specular effects. In Machine Graphics andVision, pages 137–151, 1994.
[172] Li Hector Yee, Yang Li, and Hector Yee. Spatiotemporalsensitivity and visual attention forefficient rendering of dynamic environments.ACM Transactions on Graphics, 20:39–65,2001.
[173] Sung-Eui Yoon, Sean Curtis, and Dinesh Manocha. Ray tracing dynamic scenes usingselective restructuring. InSIGGRAPH ’07: ACM SIGGRAPH 2007 sketches, page 55, NewYork, NY, USA, 2007. ACM.
[174] Kun Zhou, Qiming Hou, Rui Wang, and Baining Guo. Real-time kd-tree construction ongraphics hardware. InSIGGRAPH Asia ’08: ACM SIGGRAPH Asia 2008 papers, pages1–11, New York, NY, USA, 2008. ACM.
