ITS Updates Academic Senate December 2013 Sue Traxler Assistant Vice Chancellor for IT/CIO.
Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch,...
-
Upload
lindsay-todd -
Category
Documents
-
view
215 -
download
0
Transcript of Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch,...
![Page 1: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/1.jpg)
Differential Instant Radiosity for Mixed Reality
Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer
Institute of Computer Graphics and AlgorithmsVienna University of Technology
![Page 2: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/2.jpg)
Motivation
2Martin Knecht
![Page 3: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/3.jpg)
Motivation
3Martin Knecht
![Page 4: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/4.jpg)
Problem statement
Virtual objects should seamlessly blend into the real scene
Provide a plausible illusion
Simulation of light interaction between real and virtual objects is necessary
Solution should run at interactive to real-time frame rates
4Martin Knecht
![Page 5: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/5.jpg)
Related Work
Image Based LightingRitschel and Grosch used a HDR camera to capture environment
Hierarchical Importance Sampling, Clarberg
Real-time Global IlluminationInstant Radiosity introduced by Keller
Imperfect Shadow Maps from Ritschel et. al
Merging Real and Virtual ScenesDifferential Rendering introduced by Fournier and extended by Debevec
Differential Photon Mapping from Grosch5Martin Knecht
![Page 6: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/6.jpg)
Assumptions
Geometry of real scene known
BRDFs of real surfaces are known
Surrounding illumination is distant
6Martin Knecht
![Page 7: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/7.jpg)
Differential Instant Radiosity for MR
Idea: Combine Differential Rendering (DR) with Instant Radiosity (IR)
Why? DR adds virtual objects to real scene
IR fast method to compute GI solution in real-time
But: Two solutions are needed for DR!
not really
How? Keep track on the light paths
7Martin Knecht
![Page 8: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/8.jpg)
GI - Instant Radiosity
Use Virtual Point Lights (VPLs) to approximate global illumination
8Martin Knecht
![Page 9: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/9.jpg)
GI - Instant Radiosity
Use Virtual Point Lights (VPLs) to approximate global illumination
9Martin Knecht
![Page 10: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/10.jpg)
GI - Instant Radiosity
Use Virtual Point Lights (VPLs) to approximate global illumination
10Martin Knecht
![Page 11: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/11.jpg)
GI - Instant Radiosity
Use Virtual Point Lights (VPLs) to approximate global illumination
11Martin Knecht
![Page 12: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/12.jpg)
GI - Instant Radiosity
Use Virtual Point Lights (VPLs) to approximate global illumination
12Martin Knecht
![Page 13: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/13.jpg)
Differential Rendering
Method to add virtual objects into real scene images
Compute two global illumination solutions and add only difference to real scene image
+ Minimizes visual error from wrong BRDF approximations
- We need to calculate two GI solutions in real-time
13Martin Knecht
![Page 14: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/14.jpg)
Differential Rendering
Difference BufferGrey means no difference
GI(Real + Virtual) GI(Real)
14Martin Knecht
![Page 15: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/15.jpg)
Differential Rendering
Masked Video Image Difference Buffer
Final Image15Martin Knecht
![Page 16: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/16.jpg)
How to get GI(Real+Virtual) and GI(Real)?Keep track on the light paths
L: Lightsource
D: Diffuse light bounce
S: Specular light bounce
E: Eye
Heckberts classification
16Martin Knecht
![Page 17: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/17.jpg)
How to get GI(Real+Virtual) and GI(Real)?Keep track on the light paths
L: Lightsource
D: Diffuse light bounce
S: Specular light bounce
E: Eye
Direct illumination: LDE
Heckberts classification
17Martin Knecht
![Page 18: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/18.jpg)
How to get GI(Real+Virtual) and GI(Real)?Keep track on the light paths
L: Lightsource
D: Diffuse light bounce
S: Specular light bounce
E: Eye
Direct illumination: LDE
1st bounce diffuse indirect illumination: LDDE
Heckberts classification
18Martin Knecht
![Page 19: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/19.jpg)
Differential Instant Radiosity for MR
Calculate shading result only once
Decide on path information to which GI solution the result belongs
L[x]D[x]E
L[x]D[x]D[x]E
[x] can be „real“ or „virtual“
All „real“ then add to GI(Real+Virtual) and GI(Real)
One [x] „virtual“, only add to GI(Real+Virtual)19Martin Knecht
![Page 20: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/20.jpg)
LRDRDRE path
Differential Instant Radiosity for MR
20Martin Knecht
GI(Real + Virtual) GI(Real)
![Page 21: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/21.jpg)
LRDRDRE path
Added to both Buffers
Differential Instant Radiosity for MR
GI(Real + Virtual) GI(Real)
21Martin Knecht
![Page 22: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/22.jpg)
LRDRDVE path
Differential Instant Radiosity for MR
22Martin Knecht
GI(Real + Virtual) GI(Real)
![Page 23: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/23.jpg)
LRDRDVE path
Add only to the
GI(Real+Virtual)
Buffer
Differential Instant Radiosity for MR
GI(Real + Virtual) GI(Real)
23Martin Knecht
![Page 24: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/24.jpg)
Differential Instant Radiosity for MR
Take blocking geometry into account
24Martin Knecht
![Page 25: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/25.jpg)
LRDRDRE path
Differential Instant Radiosity for MR
25Martin Knecht
GI(Real + Virtual) GI(Real)
![Page 26: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/26.jpg)
LRDRDRE path
Virtual red wall blocks light!
Differential Instant Radiosity for MR
GI(Real + Virtual) GI(Real)
26Martin Knecht
![Page 27: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/27.jpg)
Limitations
27Martin Knecht
![Page 28: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/28.jpg)
Limitations
Double shadowing
Incosistent color bleeding
28Martin Knecht
![Page 29: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/29.jpg)
Results
Test deviceCore2 Quad CPU Q9550 at 2.8GHz
8GB RAM
NVIDIA Geforce GTX 285 with 1GB VRAM
Windows 7 64-Bit
C# with SlimDX to access DirectX 10
29Martin Knecht
![Page 30: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/30.jpg)
Results
30Martin Knecht
![Page 31: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/31.jpg)
Results
31Martin Knecht
![Page 32: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/32.jpg)
Results
32Martin Knecht
![Page 33: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/33.jpg)
Results
33Martin Knecht
![Page 34: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/34.jpg)
Our method uses instant radiosity to simulate mutual influence between objects
Direct Illumination
Indirect Illumination
Combination of Differential Rendering and Instant Radiosity
Lightsources, objects, camera can be dynamic
Cannot take all paths into account
Works reasonable fast (20 – 30 fps)
Conclusion
34Martin Knecht
![Page 35: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/35.jpg)
Consider all light paths
Add camera artifacts
Camera-based, real-time reconstruction of geometry
Real-time BRDF estimation
Future Work
35Martin Knecht
![Page 36: Differential Instant Radiosity for Mixed Reality Martin Knecht, Christoph Traxler, Oliver Mattausch, Werner Purgathofer, Michael Wimmer Institute of Computer.](https://reader035.fdocuments.us/reader035/viewer/2022062518/5697bf841a28abf838c86bab/html5/thumbnails/36.jpg)
Thank you for your attention!
36Martin Knecht