A Framework for Perceptual Studies in Photorealistic Augmented Reality
description
Transcript of A Framework for Perceptual Studies in Photorealistic Augmented Reality
![Page 1: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/1.jpg)
A Framework for Perceptual Studies in Photorealistic
Augmented RealityMartin Knecht1, Andreas Dünser2,
Christoph Traxler1, Michael Wimmer1 and Raphael Grasset3
2 HIT Lab NZ
University of Canterbury
3 HIT Lab NZ / ICG
University of Canterbury /Graz University of Technology
1 Institute of Computer Graphicsand Algorithms
Vienna University of Technology
![Page 2: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/2.jpg)
Martin Knecht 2
Introduction
MotivationPerceptual IssuesFrameworkPilot StudyLimitationsFuture WorkConclusion
![Page 3: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/3.jpg)
Martin Knecht 3
Motivation
Is this a photograph or computer generated?What visual factors affect your decision? Rademacher P., et al. (2001)
Courtesy of Rademacher et al. (2001)
![Page 4: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/4.jpg)
Martin Knecht 4
Motivation
People are very good in judging if an image looks photo-realistic or notBut process behind it not fully understood!
In augmented reality (AR) virtual objects should be rendered in a photorealistic way
Goal: A framework that allows us to study photorealistic rendering techniques
at real-time frame rates
![Page 5: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/5.jpg)
Martin Knecht 5
Perceptual Issues
Kruijff E., et al. (2010) distinguish between two types of problems:
Technological limitationsPerceptual nature
Proposed framework focuses on perceptual issues
![Page 6: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/6.jpg)
Martin Knecht 6
Perceptual Pipeline
Perceptual pipeline used to classify issuesKruijff E., et al. (2010)
Environment
Capturing
Augmentation
Display Device
User
AR Framework
![Page 7: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/7.jpg)
Martin Knecht 7
Capturing / Augmentation Stages
Capturing: Covers the process of converting the optical image to a digital one Kruijff E., et al. (2010)
Image resolution, lens distortion,exposure, color correctness,…
Augmentation: Deals with adding the virtual objects into the augmented sceneKruijff E., et al. (2010)
Registration errors, occlusion, rendering,…
![Page 8: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/8.jpg)
Martin Knecht 8
Framework
Our contribution:Framework to study photorealistic rendering techniquesInteractive tasks with global illuminationThree different rendering modesFast prototyping of experiments (XML)A pilot study to test framework
![Page 9: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/9.jpg)
Martin Knecht 9
Framework
Differential instant radiosity is used to render the scenes Knecht M., et al. (2010)
Rendering modes can be changed during run-time
Mode A: shadows / indirect illuminationMode B: shadows / indirect illuminationMode C: shadows / indirect illumination
![Page 10: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/10.jpg)
Martin Knecht 10
Rendering Mode A
Different rendering modes are supportedNo shadowsNo indirect illumination
![Page 11: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/11.jpg)
Martin Knecht 11
Rendering Mode B
Different rendering modes are supportedWith shadowsNo indirect illumination
![Page 12: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/12.jpg)
Martin Knecht 12
Rendering Mode C
Different rendering modes are supportedWith shadowsWith indirect illumination
![Page 13: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/13.jpg)
Martin Knecht 13
Pilot Study
Pilot study to evaluate our frameworkTwenty-one participants (15 male, 6 female)Age between 19 – 595 tasks
![Page 14: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/14.jpg)
Martin Knecht 14
Task 1: Estimate distance between real and virtual cubes
Pilot Study
![Page 15: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/15.jpg)
Martin Knecht 15
Task 2: Place real cube at position of virtual one
Pilot Study
![Page 16: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/16.jpg)
Martin Knecht 16
Task 3: Place virtual cube at position of real one using computer keyboard
Pilot Study
![Page 17: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/17.jpg)
Martin Knecht 17
Task 4: Grab and lift real cube
Pilot Study
![Page 18: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/18.jpg)
Martin Knecht 18
Task 5: Grab and lift virtual cube - real cube was used for tactile feedback
Pilot Study
![Page 19: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/19.jpg)
Martin Knecht 19
Pilot Study
Duration: 30 to 60 minutesInterview followed afterwardsMeasured
Distance error for tasks 1, 2, 3Time for tasks 2, 3, 4, 5
Analyzed data using non-parametric Friedman testsResult: No significant effect of rendering
technique found
![Page 20: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/20.jpg)
Martin Knecht 20
Pilot Study
![Page 21: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/21.jpg)
Martin Knecht 21
Pilot Study
![Page 22: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/22.jpg)
Martin Knecht 22
Pilot Study
Distance estimation perceived easier by 6 participants in left/right and height direction than in depth
Although no significant effect was found
![Page 23: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/23.jpg)
Martin Knecht 23
Pilot Study
Participants made heavy use of occlusion cue to place cubes in Task 2 & 3 – (19 out of 20)
![Page 24: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/24.jpg)
Martin Knecht 24
Pilot Study
7 participants preferred manipulation with a computer keyboard (Task 2 vs. Task 3).
![Page 25: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/25.jpg)
Martin Knecht 25
Limitations
No support for stereo renderingNo distance information of environmentOnly video see-through HMDs are supportedTone-mapping still needs manual fine tuningNo support for mobile devices
![Page 26: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/26.jpg)
Martin Knecht 26
Future Work
Enhance rendering frameworkSupport for stereo renderingAdd camera artifacts
New studies with new tasksTasks without occlusion informationUse chin rest to reduce movementDesign new tasks
![Page 27: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/27.jpg)
Martin Knecht 27
Conclusion
We proposed a research test-bed for perceptual studies – still work in progress
Different rendering modes allow new kind of experiments
We showed the results of a pilot study using our framework
![Page 28: A Framework for Perceptual Studies in Photorealistic Augmented Reality](https://reader035.fdocuments.us/reader035/viewer/2022081520/56815e95550346895dcd256d/html5/thumbnails/28.jpg)
Thank you for your attention!What features would you like to have in such a framework?
28Martin Knecht