Low Latency Rendering with Dataflow Architectures

EngD Project

Sebastian Friston

Supervisor: Anthony Steed

Motivation: latency in virtual environments

• Latency is the time between a user’s action and the response to this action

• Latency in VR reduces sensory coherence– This coherence is key for creating a sense of presence

• Latencies of 10-16 ms have been shown to have a significant negative effect– Latencies under 5 ms can be detected though

Motivation: latency in GPUs

(Mine, 1993)

Motivation: latency in GPUs

The Rendering Continuum

(Zwicker, et al. 2000)

Dataflow Processing

Dataflow Processing

• True-Parallel Execution• No scheduler• Space taken is

proportional to all possible operations

• Resources are close by

Light Field Rendering

(Gortler, et al. 1996)

Light Field Renderers

• Have been implemented on GPUs and FPGAs

• Captured with cameras or synthesized

• Most practical applications have been to use them as a cache for volume renders

• Current implementations are limited by memory

(Birklbauer, et al 2013)

(Regan, et al 1999)

Hardware Accelerated Light Field Renderer

Validation and Complications

• Validate our renderer with latency interaction experiment– Possibly investigate phenomena where scale is hard to

judge in VR– Possibly continue studies into detection during head

rotations

• Display will be difficult– Current displays have latencies of ~6 ms

Thank you

(Birklbauer, et al. 2013)

References

• Mine, M. R. (1993). Characterization of end-to-end delays in head-mounted display systems. University of North Carolina at Chapel Hill

• Zwicker, M., Gross, M., & Pfister, H. (2000). A Survey and Classification of Real Time Rendering Methods.

• Birklbauer, C., Opelt, S., & Bimber, O. (2013). Rendering Gigaray Light Fields. Computer Graphics Forum, 32(2pt4), 469–478. doi:10.1111/cgf.12067

• Gortler, S. J., Grzeszczuk, R., Szeliski, R., & Cohen, M. F. (1996). The lumigraph. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques - SIGGRAPH ’96 (pp. 43–54). New York, New York, USA: ACM Press. doi:10.1145/237170.237200

• Regan, M. J. P., Miller, G. S. P., Rubin, S. M., & Kogelnik, C. (1999). A real-time low-latency hardware light-field renderer. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’99 (pp. 287–290). New York, USA: ACM Press. doi:10.1145/311535.311569