Advanced Topics in Virtual Reality - TU Wien...Anton L. Fuhrmann 2016 In optics, the exit pupil is a...

Anton L. Fuhrmann

2017

1

Advanced Topics in Virtual RealityDisplay Devices

http://tinyurl.com/k8sfdxr

http://tinyurl.com/k8sfdxr

Anton L. Fuhrmann

2016

Advanced Topics in VR

Display Devices

Stereoscopic Output

Desktop Screens w/Glasses

Projection w/Glasses

Head-Mounted Displays

Autostereoscopic Displays

Volumetric Displays

Anton L. Fuhrmann

2016

Classification of stereoscopic displays


Anton L. Fuhrmann

2016

Classification of AR displays


4

Anton L. Fuhrmann

2016


head-mounted (!) helmet, goggles, clips, ...

one or two displays directly in front of eyes

two displays perfect stereo channel separation!

display technology:

CRT

LCD

laser (retinal displays)

projector

type: immersive, see-through, video see-through

Head-Mounted Display (HMD)

Anton L. Fuhrmann

2016

Human Field of View


6

[Bimber2005]

Anton L. Fuhrmann

2016

In optics, the exit pupil is a virtual aperture in an optical system. Only rays which pass through this virtual aperture can be seen by the user.


Exit Pupil

Objective Exit Pupil

Eye Relief

Anton L. Fuhrmann

2016


Vergence & Accomodation

Vergence–accommodation conflicts hinder visual performance and

cause visual fatigue

David M. Hoffman; Ahna R. Girshick; Kurt Akeley; Martin S. Banks

Journal of Vision March 2008, Vol.8, 33. doi:10.1167/8.3.33

Anton L. Fuhrmann

2016


1960 Heilig

Head-Mounted Display History

1963 Hall, Miller

Anton L. Fuhrmann

2016


Head-Mounted Display History

1969 Sutherland

•vector display: “3000 lines

at 30 frames per second”

•ultrasound & mechanical

tracking

•CRT see-through display

Anton L. Fuhrmann

2016


optical see-through

semi-transparent mirror

overlay over real world

brightness problem

occlusion problem

video see-through

real world via camera composited

occlusion solvable

only video resolution of real world (~HD)

Head-Mounted Display (HMD)

Anton L. Fuhrmann

2016


optical see-through HMD

[Haller2007]

Anton L. Fuhrmann

2016



display via half-silvered mirror as

overlay (“add”) over reality

Advantage: undistorted view of

reality in realtime an wide FoV

Disadvantage: no complete

occlusion, only visible against

darker background

Anton L. Fuhrmann

2016



other optical configurations

(3D VIS Lab, University of Arizona - "Head-

Mounted Display Systems" by Jannick Rolland

and Hong Hua)

Anton L. Fuhrmann

2016


pinlight see-through HMD

defocused pinlights act as

“projectors”, projecting a virtual

image through an LCD on the

retina

Advantage: wide FoV

Disadvantage: preprocessing

dependent on eye position, eye

positions has to be measured

Maimone, A., Lanman, D., Rathinavel, K., Keller, K., Luebke, D., Fuchs, H. 2014. Pinlight Displays:

Wide Field of View Augmented Reality Eyeglasses using Defocused Point Light Sources.

ACM Trans. Graph. 33, 4, Article 89 (July 2014), 11 pages. DOI = 10.1145/2601097.2601141

http://doi.acm.org/10.1145/2601097.2601141.

Anton L. Fuhrmann

2016


pinlight see-through HMD

Maimone, A., Lanman, D., Rathinavel, K., Keller, K., Luebke, D., Fuchs, H. 2014. Pinlight Displays:

Wide Field of View Augmented Reality Eyeglasses using Defocused Point Light Sources.

ACM Trans. Graph. 33, 4, Article 89 (July 2014), 11 pages. DOI = 10.1145/2601097.2601141

http://doi.acm.org/10.1145/2601097.2601141.

Anton L. Fuhrmann

2016


diffractive see-through HMD

[„Diffractive and Holographic Optics as

Optical Combiners in Head Mounted

Displays“, Bernard C. Kress]

Vuzix / Nokia Waveguide

diffractive combiner (with

laser pico projector)

Anton L. Fuhrmann

2016


video see-through HMD

[Haller2007]

Anton L. Fuhrmann

2016


video see-through HMD

camera delivers image to display

no direct view of reality

Advantage: real image can be

manipulated, too!

Disadvantage: time lag, parallax

between eye and camera view

point

But parallax-free design

possible!

Anton L. Fuhrmann

2016


parallax-free video see-through

HMD

optics are designed so

that the (reflected)

camera viewpoint is in

the users pupil

video:

[State2005]

Anton L. Fuhrmann

2016


parallax-free video see-through

HMD

free – form prisms

allow for an extremely

compact construction

[Inoguchi2008]

Anton L. Fuhrmann

2016


Addvisor (SAAB)

Commercial HMDs

i-glassesST40 (Kaiser)

datavisor (nvision)

glasstron (Sony)

Anton L. Fuhrmann

2016


Commercial HMDs: Panoramic HMD

www.Sensics.com

claims:

Panoramic field of view: from 82° to 180°

diagonal

A modular, upgradeable design

High resolution: Up to 4200x2400 pixels

per eye (2400x1720 effective)

Ease of use: weighing less than 1 kg (2

lbs.), open-frame design: comfortable and

stays cool

http://www.sensics.com/

Anton L. Fuhrmann

2016



the field of view is considerable larger than other HMDs’:

delivering a better sense of immersion

Anton L. Fuhrmann

2016



The wide field of view is made possible by using not

one LCD but several, which in combination with

special optics tile seamlessly:

Anton L. Fuhrmann

2016


Retinal HMD

image exists on

display surface, is

viewed by eye

image exists only

on retina

Anton L. Fuhrmann

2016


since the laser beam is extremely thin, even tiny particles throw shadows on the retina

“eye floaters”:

Retinal HMD

http://en.wikipedia.org/wiki/Floater

Anton L. Fuhrmann

2016


uses laser to directly project on users retina

Advantages: bright, always focussed, see-through

Disadvantages: very sensitive to impurities in the eye!

Retinal HMD (NOMAD, Microvision)

Anton L. Fuhrmann

2016


Preliminary specs:

Display

High resolution display is the equivalent of a 25 inch (64cm) high definition screen from 8 feet (2.4m) away (≈15° diagonal)

Camera

Photos - 5 MP

Videos - 720p

Audio

Bone Conduction Transducer

Connectivity

Wifi - 802.11b/g

Bluetooth

Google Glass

Anton L. Fuhrmann

2016


HTC Vive

Anton L. Fuhrmann

2016


Oculus Rift

Anton L. Fuhrmann

2016

shader corrects geometric & chromatic distortions:


Oculus Rift

Anton L. Fuhrmann

2016

Oculus Rift HTC Vive

Display OLED OLED

Resolution (overall) 2160 x 1200 2160 x 1200

Refresh rate 90Hz 90Hz

Field of View 110° 110°

Tracking area 1.5m × 1.5m 4m × 4m

Sensors/Trackers Accelerometer, gyroscope,

magnetometer,

Constellation tracking

camera

Accelerometer, gyroscope,

front-facing camera,

Lighthouse laser tracking

system,


Rift vs. Vive

Anton L. Fuhrmann

2016

Micro lenses in front of segmented views:


Light Fields HMD

Anton L. Fuhrmann

2016

Flat (~10mm) optical element:


Light Fields HMD

Anton L. Fuhrmann

2016

Depth of Field:


Light Fields HMD

Anton L. Fuhrmann

2016


Light Fields HMD (video)

Anton L. Fuhrmann

2016

Advantages:

• depth-of-field

• thin optics

Disadvantages:

• greatly reduced resolution

• extremely dependent on exit pupil


Light Fields HMD

"Near-Eye Light Field Displays"

Douglas Lanman (NVIDIA), David Luebke (NVIDIA), in ACM Transactions on Graphics (TOG),

Volume 32 Issue 6, November 2013 (Proceedings of SIGGRAPH Asia), November 2013

F. Huang, K. Chen, G. Wetzstein. “The Light Field Stereoscope: Immersive Computer

Graphics via Factored Near-Eye Light Field Displays with Focus Cues”, ACM SIGGRAPH

(Transactions on Graphics 33, 5), 2015.

https://youtu.be/YJdMPUF8cDM

https://research.nvidia.com/users/douglas-lanman

https://research.nvidia.com/users/david-luebke

http://dl.acm.org/citation.cfm?id=2508366

https://youtu.be/YJdMPUF8cDM

Anton L. Fuhrmann

2016


uses projector to display from users viewpoint on (retro-reflective) environment

Advantages: „correct“ occlusion

Disadvantages: heavy, varying focus distance, stereo separation depends on retro-reflection quality

Head-Mounted Projectors

Anton L. Fuhrmann

2016


Head-Mounted Projectors

simplified head-mounted projector set-up

example prototypes

(note „glowing eyes“)

[Furht2011]

Anton L. Fuhrmann

2016


head attached display: VR

telescopeConsists of a video camera and monitor

Very rugged, immediately usable by

untrained users

mechanical tracking in the joint allows

easy video augmentation (precise & fast

rotation sensor)

https://www.igd.fraunhofer.de/en/Institut/Abteilungen/VRAR/Projekte/AR-Telescope

http://www-past.igd.fraunhofer.de/igd-a4/technologie-services/ar-teleskop/

Anton L. Fuhrmann

2016


head attached display: zacturn 2.0comparable to “VR telescope”:

Austrian development: www.zkoor.at

Anton L. Fuhrmann

2016


head attached display: zacturn 2.0

Anton L. Fuhrmann

2016


head attached display: Boom

Fakespaceboom

Osaka University „ELMO“

Anton L. Fuhrmann

2016

Hand-held Displays

Tablet PCs

smartphones

essentially the same as video HMDs

contain enough sensors for many AR applications:GPS

compass

acceleration

camera(s)


46

Anton L. Fuhrmann

2017

47

Projection Displays

Anton L. Fuhrmann

2016


Classification of Projection Technology

Cathod Ray Tubes (CRT)

Liquid crystal (LCD)

Micro-Mirrors (DLP)

Reflective LCD (LCOS, D-ILA)

exotic and little-used devices(Laser, Eidophor)

Anton L. Fuhrmann

2016


„Cathode Ray Tube“ („Kathodenstrahlröhre“)

Same technology as classic TV-set

Three different tubes (RGB)

Analog technology

Fast (120Hz-180Hz frame-rate active stereo)

Low luminance (typically 1000 ANSI lm)

Good color reproduction

Black is really black!

Expensive, but extremely good images

CRT Projector

Difficult to adjust

Anton L. Fuhrmann

2016


CRT Projector

Anton L. Fuhrmann

2016


Calibrating them is a lengthy and tedious procedure:

STEP ONE: VERIFY THE CORRECT SCREEN/PROJECTOR DIMENSIONS

STEP TWO: VERIFY THE CORRECT SCANNING POLARITY

STEP THREE: CENTER THE GREEN IMAGE IN THE PICTURE TUBE

STEP FOUR: CENTER THE GREEN IMAGE ON THE SCREEN

STEP FIVE: ADJUST THE OPTICAL FOCUS

STEP SIX: ADJUST THE TUBE “FLAPPING”

STEP SEVEN: ADJUST THE GREEN REFERENCE IMAGE

STEP EIGHT: ADJUST THE GREEN REFERENCE IMAGE OVERSCAN

STEP NINE: MATCH THE RED IMAGE TO THE GREEN REFERENCE

STEP TEN: MATCH THE BLUE IMAGE TO THE GREEN REFERENCE

STEP ELEVEN: CENTERING THE USER POSITION CONTROLS

(from a manual)

and then: do it all again for the second projector!

CRT Projector

Anton L. Fuhrmann

2016


LCD Projector

„Liquid Crystal Display“ („Flüssigkristall“)

Transmissive LCD panel(s) with lamp

Slight color-dependent polarization troubles with polarized stereo!

Mediocre color reproduction (no gamma curve)

Higher luminance (>1000 ANSI lm)

only about 50% of panel surface active

Anton L. Fuhrmann

2016


DLP (Micro-Mirror) Projector

„DLP™“ Digital Light Processing

Micro-Mechanical Chips byTexas Instruments

about 89% of panel surface active

but visible pixel structure

inexpensive, but variable color reproduction

Anton L. Fuhrmann

2016


DLP Projector

micro-mechanic:

tiltable mirrors

two positions, switched by electrostatic force

Anton L. Fuhrmann

2016


DLP Projector

tiltable mirrors, electron microscopic view

Anton L. Fuhrmann

2016


DLP Projector

mirrors produce on/off pattern only

modulate on-to-off ratio to generate grayscale

Anton L. Fuhrmann

2016


DLP Projector

inexpensive single-chip projectors need color filter wheel to generate colors:

in many cases, these wheels contain „white“ as additional color, thereby significantly enhancing black/white contrast, but impairing color reproduction

Anton L. Fuhrmann

2016


DLP Projector

Only expensive three-chip projectors support faithful color reproduction:

Since micro-mirrors are extremely fast, some of these expensive projectors support output frame rates of >100Hz and thereby active stereo

Examples:

BARCO Galaxy

Christie Mirage

Anton L. Fuhrmann

2016


DLP LED/Laser Projector

Use combined LED/Laser light source instead of mercury lamp

Advantage:

Higher efficiency

Longer lamp life (~20.000h)

No color wheel, since LED/Laser can be pulsed

Disadvantage:

Still not very bright (~3500lm)

Anton L. Fuhrmann

2016


reflective LCD Projector

„LCoS“ Liquid Crystal on Silicon

„D-ILA“ Direct Drive Image Light Amplifier

high resolutions (up to QXGA, 2048 x 1536)

almost no pixel structure (93% area usable)

good colour reproduction

medium luminance

Anton L. Fuhrmann

2016



The switching elements are placed behind the liquid crystal, so almost no active display area is lost (only 7%)

Anton L. Fuhrmann

2016



The light from the lamp is reflected from the surface of the display element:

Anton L. Fuhrmann

2016



Using three elements and some prisms, we get color output:

Anton L. Fuhrmann

2016



LCD

LCD

prism

Anton L. Fuhrmann

2016


Autostereoscopic Display Devices

„All devices which do NOT needspecial glasses and can beviewed with the naked eye“

Anton L. Fuhrmann

2016


Parallax Stereo with Lenticulars

Lenticular lenses (cylinders) send different images in different directions.

Similar to “3D” postcards and stickers.

Anton L. Fuhrmann

2016


Parallax Stereo with Lenticulars

A raster of cylindrical lenses in front of an LCD shows different rows/colums to the users eyes:

Anton L. Fuhrmann

2016


Parallax Barrier

A simple parallax barrier works like this:

A parallax barrier shields a display screen. The distance between screen and barrier, as well as the resolutions of both are calculated to correctly display left pixels (green) only to the left eye and vice versa.

Anton L. Fuhrmann

2016


Slanted

Parallax BarrierA simple approach extremely reduces the horizontal resolution.

+

= =

Better: treat each LCD-triade as three independent pixels ⇒ “triples” horizontal resolution

+

use slanted barrier, otherwise one view-slice gets only one primary color!

http://csc.lsu.edu/~kooima/pdfs/Kooima-VR07.pdf

Anton L. Fuhrmann

2016


Image calculated for SPB:

Slanted Parallax Barrier

original view detail

parallax view detail

Anton L. Fuhrmann

2016


Advantages vs. simple approach or lenticular lenses:

“higher” resolution (distributed over vertical resolution too)

less view-dependent artifacts than lenticulars

less expensive to manufacture than lenticulars

Disdvantages vs. lenticular lenses:

lower contrast, since all light not coming through a slot in the barrier is absorbed

Slanted Parallax Barrier

Anton L. Fuhrmann

2016


Adaptive Parallax Barrier

Ken Perlin built the prototype of a display with adjustable parallax barrier (essentially another vertically striped LCD in front of the display:

http://mrl.nyu.edu/~perlin/courses/fall98/projects/autostereo.html

Anton L. Fuhrmann

2016


Parallax Stereo

Advantages:

no glasses

multi-user, when more than 2 viewing zones

Disadvantages:

lenticular system exhibits color artifacts

works in praxis only horizontally

reduces spatial resolution of display

mediocre separation

Anton L. Fuhrmann

2016


Active Stereocopic Devices

Means stereo de-multiplexed by „active“ glasses, with electronics:

time-multiplex: left/right/left/right…

needs fast display (CRT, DLP)

needs “active” glasses with shutters, whichhide the wrong images synchronization with source needed!

works good for back-projection

Anton L. Fuhrmann

2016


Active Stereo Devices

infrared emitter

LCD-shutter

glasses

pulsed infrared

light sync signal

or RF or

„DLP-Link“

fast display

(monitor or projector)

>100frames/s

Barco “BARON”

virtual table(backprojection with

CRT projector)

Anton L. Fuhrmann

2016



single DLP projector

resolution : 1400x1050

Brightness (Lumens) : 2400 ANSI

“contrast”: 3500:1

Weight: 3,4 kg

Size (cm) (HxWxD): 278 x 278 x 110 mm

Price : ≈ 20,000€

Hfreq: >120Hz

“Dual-head”: two separate inputs for R/L

compatible with all graphic cards with two

outputs!

Projectiondesign:

“F10 AS3D”

Anton L. Fuhrmann

2016



single DLP projector

resolution : Full HD (1920x1080)

Brightness (Lumens) : 2500 ANSI lm

Full On/Off: 3200:1

Weight: 2.2 kg

3D (Stereo) Full HD @ 24Hz

Price : ≈ 900€ !

Optoma DH1017

Anton L. Fuhrmann

2016



1 DLP projector

resolution : 1280x800

Brightness (Lumens) : 3200 ANSI lm

Full On/Off: 2300:1

Weight: 7 kg

Price : ≈ 900€ !

Ultra short throw distance:

DELL s500 / NEC NP-U300x

200cm diagonal

@50cm distance

Anton L. Fuhrmann

2016


Ultra Short Throw Projection

Main advantages is that front projection for surround display possible:

(Almost) no shadowing by user compared to “classical” front projection

Better luminance distribution than rear projection (low vignetting)

Anton L. Fuhrmann

2016


Ultra Short Throw Projection

Main principle: free-form (not spherical!) mirror in beam path:

© Ricoh

Anton L. Fuhrmann

2016


Volumetric Display

The problem of wrong accommodation can be solved by generating “real” 3D displays, where the images are formed at the correct points in space, not by binocular viewing of stereo images on a flat plane:

Volumetric displays create 3D imagery via the emission, scattering, or relaying of illumination from well-defined regions in (x,y,z) space

Anton L. Fuhrmann

2016


Classification of Volumetric D.

Ochiai et.al. 2015

Anton L. Fuhrmann

2016


Swept-volume display

LEDs in a cubic grid:

simpel & bad!

Anton L. Fuhrmann

2016



Projection on a fast rotating screen persistence of vision

The screen has a form that allows the projection to cover all points in the viewing volume

Anton L. Fuhrmann

2016



The rotating screen has a form that allows the projection to cover all points in the viewing volume

Not exactly new:

Apparatus for forming three dimensional

images. Leighton Homer Surbeck, 1940

Spherical Spiral Display. D.W. Perkins, 1962

Anton L. Fuhrmann

2016



Rotating emitters:

e.g. a screen rotatingaround a vertical axis

Anton L. Fuhrmann

2016

Uses the plasma emission phenomenon near the focal point of focused laser light


Volumetric Plasma Display

Hidei Kimura, Taro Uchiyama, and Hiroyuki Yoshikawa.

2006. Laser produced 3D display in the air. In ACM

SIGGRAPH 2006 Emerging technologies (SIGGRAPH

'06). ACM, New York, NY, USA, , Article 20 .

DOI=http://dx.doi.org/10.1145/1179133.1179154

Anton L. Fuhrmann

2016

Uses the plasma emission phenomenon near the focal point of focused laser light


Volumetric Plasma Display#2

Anton L. Fuhrmann

2016


Volumetric Plasma Display#2:

Fairy Lights

Yoichi Ochiai, Kota Kumagai, Takayuki Hoshi, Jun Rekimoto, Satoshi Hasegawa, Yoshio Hayasaki, Fairy Lights in

Femtoseconds: Aerial and Volumetric Graphics Rendered by Focused Femtosecond Laser Combined with

Computational Holographic Fields, http://arxiv.org/abs/1506.06668 (ArXiv Preprint: *Submitted to ACM Transactions

on Graphics (via Acceptance with major revision in ACM SIGGRAPH 2015)

http://arxiv.org/abs/1506.06668

Anton L. Fuhrmann

2016


Fairy Lights

Anton L. Fuhrmann

2016


Rotating Mirror Display

(not really volumetric, only horizontal parallax) http://gl.ict.usc.edu/Research/3ddisplay/

Anton L. Fuhrmann

2016


Rotating Mirror Display

http://gl.ict.usc.edu/Research/3ddisplay/

Uses an anisotropic holographic diffuser bonded onto a first surface mirror.

Horizontally, the mirror is sharply specular to maintain a 1.25 degree separation between views.

Vertically, the mirror scatters widely so the projected image can be viewed from multiple heights.

only horizontal parallax

but correct occlusion!

Anton L. Fuhrmann

2016


Next: Stereoscopy Methods

Advanced Topics in Virtual Reality - TU Wien...Anton L. Fuhrmann 2016 In optics, the exit pupil is a...

Documents

Transcript of Advanced Topics in Virtual Reality - TU Wien...Anton L. Fuhrmann 2016 In optics, the exit pupil is a...