Real-Time Relighting Digital Image Synthesis Yung-Yu Chuang 1/10/2008 with slides by Ravi...
-
Upload
gwendolyn-byrd -
Category
Documents
-
view
214 -
download
0
Transcript of Real-Time Relighting Digital Image Synthesis Yung-Yu Chuang 1/10/2008 with slides by Ravi...
Real-Time Relighting
Digital Image SynthesisYung-Yu Chuang1/10/2008
with slides by Ravi Ramamoorthi, Robin Green and Milos Hasan
Realistic rendering
• We have talked about photorealistic rendering for complex materials, complex geometry and complex lighting. They are realistic but slow.
Real-time rendering
• Its goal is to achieve interactive rendering with reasonable quality. It’s important in many applications such as games, visualization, computer-aided design, …
Real-Time relighting
• Lighting is the process of adjusting lights. It is an important but time-consuming step in animation production pipeline.
• Relighting algorithms for two kinds of lights– Distant environment lights– Near-field lights for production
Relighting algorithms for distant environment lights
Natural illumination
People perceive materials more easily under natural illumination than simplified illumination.
Images courtesy Ron Dror and Ted Adelson
Natural illumination
Rendering with natural illumination is more expensive compared to using simplified illumination
directional source natural illumination
Reflection maps
Blinn and Newell, 1976
Environment maps
Miller and Hoffman, 1984
HDR lighting
Examples of complex environment light
Examples of complex environment light
Direct lighting with complex illumination
)ωp,( ooL )ω,p( oeL
iiiio ωθcos)ωp,()ω,ωp,(2
dLf is
)ωp,( oB iiiio ωθcos)ωp,()ω,ωp,(2
dLf ds)ω( opB iiiiω, ωθcos)ω()ω(
2 odLf ds p
p q
Function approximation
• G(x): the function to approximate• B1(x), B2(x), … Bn(x): basis functions• We want
• Storing a finite number of coefficients ci gives an approximation of G(x)
)()(1
xBcxG i
n
ii
Function approximation• How to find coefficients ci?
– Minimize an error measure• What error measure?
– L2 error
• Coefficients
2])()([
2 I i
iiL xBcxGE
X
iii dxxBxGBGc )()(
• Basis Functions are pieces of signal that can be used to produce approximations to a function
1c
2c
3c
Function approximation
• We can then use these coefficients to reconstruct an approximation to the original signal
1c
2c
3c
Function approximation
• We can then use these coefficients to reconstruct an approximation to the original signal
xBcN
iii
1
Function approximation
Orthogonal basis functions
• Orthogonal Basis Functions– These are families of functions with special
properties
– Intuitively, it’s like functions don’t overlap each other’s footprint• A bit like the way a Fourier transform breaks
a functions into component sine waves
ji
jidxxBxB ji 0
1
Integral of product
iiij
i jiji
i jjjii
GFdxgfdxxBxBgf
dxxBgxBfdxxGxF
ˆˆ)()(
)()(
dxxGxFI
i
ii xBfxF )( j
jj xBgxG )(
)ω( opB iiiiω, ωθcos)ω()ω(2 o
dLf ds p
Basis functions
• Transform data to a space in which we can capture the essence of the data better
• Spherical harmonics, similar to Fourier transform in spherical domain, is used in PRT.
Real spherical harmonics
• A system of signed, orthogonal functions over the sphere
• Represented in spherical coordinates by the function
where l is the band and m is the index within the band
0
0
0
,cos
,cossin2
,coscos2
,00 m
m
m
PK
PmK
PmK
y
ll
ml
ml
ml
ml
ml
Real spherical harmonics
Reading SH diagrams
–+
Not thisdirection
Thisdirection
Reading SH diagrams
–+
Not thisdirection
Thisdirection
The SH functions
00y
11y
11y
12y
22y
02y
12y2
2y
The SH functions
Spherical harmonics
Spherical harmonics
-1-2 0 1 2
0
1
2
( , )lmY
xy z
xy yz 23 1z zx 2 2x y
1
m
l
SH projection
• First we define a strict order for SH functions
• Project a spherical function into a vector ofSH coefficients
S
ii dssysfc
mlli 1
SH reconstruction
• To reconstruct the approximation to a function
• We truncate the infinite series of SH functions to give a low frequency approximation
2
0
~ N
iii sycsf
Examples of reconstruction
An example
• Take a function comprised of two area light sources– SH project them into 4 bands = 16 coefficients
2380042508370317000106420
27800417009400908093006790
3291
.,,.,.,.,.,.,.,,.,,.
,.,.,.,.
Low frequency light source
• We reconstruct the signal– Using only these coefficients to find a low frequency
approximation to the original light source
Harr wavelets• Scaling functions (Vj)
• Wavelet functions (Wj)
• The set of scaling functions and wavelet functions forms an orthogonal basis
Harr wavelets
Example for wavelet transform
• Delta functions, f=(9,7,3,5) in V2
Wavelet transform
• V1, W1
Example for wavelet transform
• V0, W0 , W1
Example for wavelet transform
Quadratic B–spline scaling and wavelets
2D Harr wavelets
Example for 2D Harr wavelets
Applications
19%5% L2
1%15% L2
3%10% L2
Relighting algorithms for animation production
Relighting for production
• Lighting is a time-consuming process.• Artists adjust lighting parameters and wait for
a couple of hours or days to get feedback.• Local shading with complex scene and many li
ghts• Interactive relighting
– Interative visual eedback– Fixed scene and camera– Lower quality – Scalable with sene complexity and number of light
s
Deep framebuffer• Gershbein and Hanrahan, SIGGRAPH 2000
Deep framebuffer
Deep framebuffer
LPICS
• Pixar, SIGGRPH 2005. A practical realization for the deep framebuffer approach on GPUs
LPICS0.1s
Final renderer2,000s
video
Lightspeed• ILM, SIGGRAPH 2007• An even more practical system with automatic
shader conversion. (2.7s v.s. 57m)
Direct-to-indirect transfer
• Hasan et. al. SIGGRAPH 2006• Deep framebuffer approaches only support loc
al shading, but not indirect lighting
direct lighting With indirect lighting
Concept
• Distribute gather samples on scene surfaces
Concept
• Direct illumination on both gather samples and view samples
Concept
• Inter-reflections between gather samples
Concept
• Final gather on view samples
Inter-reflections between gather samples
gather sample
gather sample
Inter-reflections between gather samples• Assume all gather samples are diffuse
Inter-reflections between gather samples
Inter-reflections between gather samples
Final gathering
Direct on gather
Indirect on view
Final
Transfer
matrix
Direct on view
Concept
Scene: Still Life
Precomputation: 1.6 hours
11.4 – 18.7 fps Polygon: 107k
Scene: Temple
Precomputation: 2.5 hours
8.5 – 25.8 fps Polygon: 2M
Scene: Hair Ball
Precomputation: 2.9 hours
9.7 – 24.7 fpsPolygon: 320k
Scene: Sponza Atrium
Precomputation: 1.5 hours
13.7 – 24.9 fps Polygon: 66k
Comparison
DTI: 8-25 fps (2.5 hr precomputation)
Monte Carlo path tracer: 32 hours