Immersive Virtual Reality

Immersive Virtual RealitySébastien Kuntzhttp://cb.nowan.net/blogSebastien.Kuntz@nowan.net

Last updateNovember 2012

Friday, November 16, 12

License

You are free:

• to Share — to copy, distribute and transmit the work

• to Remix — to adapt the work

• to make commercial use of the work

Under the following conditions:• Attribution — You must attribute the work in the

manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

• You must attribute “Immersive VR introduction” to (with this link):• Sébastien Kuntz - http://cb.nowan.net/blog

”Immersive Virtual Reality Introduction” by Sebastien Kuntz is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/

• With the understanding that:• Waiver — Any of the above conditions can be

waived if you get permission from the copyright holder.

• Public Domain — Where the work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.

• Other Rights — In no way are any of the following rights affected by the license:◦ Your fair dealing or fair use rights, or other

applicable copyright exceptions and limitations;◦ The author's moral rights;◦ Rights other persons may have either in the

work itself or in how the work is used, such as publicity or privacy rights.

• Notice — For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to this web page.

Definitions

VR Systems

Applications

Techniques

WHO AM I ?• Founder editor of

• http://www.imin-vr.com

• AFRV - French VR association: Board member - http://www.af-rv.fr

• VR Geeks association: President - http://www.vrgeeks.org

• Blog: http://cb.nowan.net/blog

• Twitter : @Cb_VRGeek

• 10 years in VR

• Virtools / Dassault Systèmes

• SNCF - French railwaysFriday, November 16, 12

What is VR for you ?

Immersive Virtual Reality

Presence in a virtual world

(c) eMagin(c) Barco

Immersive VR = iVR

iVR is the science and technology required for a user to feel present, via perceptive, cognitive

and functional immersion and interaction, in a (computer) generated environment.

Presence

Cognitive presence

Perceptive presence

Results

Natural interactions

Natural reactions

What is VR ?

Realtime 3D != VR

Stereoscopy != VR

3D Tracking != VR

Haptics != VR

3D Sound != VR

3D Interactions != VR

But everything combined can be VR

RidiculousIf it’s real VR, you probably

look ridiculous

Already changing world in industry and research

Democratization

Are we in the future ? No ! We’re still in prehistory !

WE can make a difference, invent systems, applications, businesses, interactions

Why now ?

Measure presence ?

Can’t objectively measure presence (qualia)

What you can measure is if people respond to this Virtual Environment as if it is real, which is a result of presence

Compare real-life behavior vs virtual

If avatar: brain zone activates when user accepts the avatar as real

Immersion parametersGraphics framerate

Extent of tracking

Tracking latency (measure?)

Quality of the images

Field of view

Plausibility of behaviors

Range of sensory modalities

... Friday, November 16, 12

Paul Milgram

VR Systems

Projector based VR Systems

http://www.jamstec.go.jp/esc/research/Perception/vr.en.htmlFriday, November 16, 12

(c) Sébastien Kuntz

http://www.youtube.com/watch?v=Hb29bLHEjUgClarte - SAS Cube

Immersia 3

(c) Sébastien KuntzFriday, November 16, 12

Immersia 3 - IRISA Rennes

http://www.youtube.com/watch?v=cTd5DLe8KD0

Calit2 - StarCave

http://www.scivee.tv/node/3648http://www.calit2.net/newsroom/release.php?id=1383

Projector basedReality center - HoloStage

(c) Arnaud MutheletFriday, November 16, 12

Projector based - Workbench

http://liris.cnrs.fr/~sbrandel/en/research/VR/LSIIT-IGG

Projector based

Workbench © PSA Peugeot Citroën Mediatheque.

Clarte R-Screen

http://www.youtube.com/watch?v=AcV5ma5nEVgFriday, November 16, 12

Panoscope 360°

(c) MatsushitaFriday, November 16, 12

La GéodeFriday, November 16, 12

http://www.youtube.com/watch?v=rNsFSVhZi9wFriday, November 16, 12

ToobImmersive dome

http://youtu.be/N6LiLaNOHHU

Barco - RP-360

• 13 projectors

• Diamètre: 3.4m

Barco - RP-360

http://www.youtube.com/watch?v=3EbrQwpOx24

6 users: stereo + tracking

http://www.uni-weimar.de/cms/medien/vr/research/display-systems/c1x6-a-stereoscopic-six-user-display-for-co-located-collaboration-in-shared-virtual-environments.html

Weimar University (DE)Friday, November 16, 12

6 usersstereoscopy + tracking

• video

Screen based

Full-scale NexCAVE developed and deployed by Calit2 for Saudi Arabia's King Abdullah University of Science and Technology - http://www.calit2.net/newsroom/release.php?id=1743

Ultra-Short throw projectors

• Sanyo PDG-DWL2500

• 1280x800

• 2500 lumens

• Canon LV-8235 UST

Ultra-Short throw projectors

http://www.youtube.com/watch?v=VSblSrTRJh0

TV-3D - 154”

(c) Sébastien Kuntz

Projector based VR Systems(-)

(Very) expensive !

Lots of space ! ( front and behind, AC... )

Not 360°

Cluster hard to program !

+/- Collaborative

Not much to wear

High resolution, big FOV

HMDHead mounted displays

Vuzix•640x480•32° FOV

•852x480•32° FOV

eMagin Z800

800x600

40° FOV

HMZ-T1

1280x1024

Sensics

piSight

Multiple screens

zSight

1280x1024

70° FOV

nVisor SX60

1280x1024

60° FOV

nVisor SX111

1280x1024

111° FOV

Fakespace Labs

150° horizontal FOV

1600x1200

Progressive

• Developer kit

• 90° horizontal FOV

• 1280x800

• 640x800 per eye

OCULUS RIFT

HMD(-)

Low resolution, low FOV

Must wear, adjust

No peripheral vision

Lightweight

Compact

Easier to program

360° tracking

SummaryProjector/screen based

Cave, holostage, workbench

Collaborative, high FOV

Expensive, Complex

“Cheap”

Low Fov/resolution

Depth perception

How do you perceive depth ?

Stereoscopy ?

http://www.youtube.com/watch?v=Jd3-eiid-UwJohnny Lee - Head Tracking with a wiimote

Stereoscopy ?

http://www.youtube.com/watch?v=nRYybo2WNwU

Esiea Virtual ShowcaseFriday, November 16, 12

Stereoscopy

You perceived depth without stereoscopy !

Depth perceptionStereoscopic cues :

eye accomodation

eye convergence angle

disparity between left and right images

Monoscopic cues :

perspective

atmospheric artifacts (fog)

relative sizes

occlusion

parallax

shadows

Parallax can be more important for depth perception!Stereoscopy is important for size and distance

evaluationFriday, November 16, 12

Stereoscopy

• Useful for

• Relative distance perception

• Detection of objects

• Discrimination between two objects

• Localisation

• But...

• Fusing problems

• Headaches

• Visual fatigue

• Accommodation / Vergence dissociation

Different stereoscopic separation schemes

Active shutter glasses

Polarized glasses

Anaglyph

Infitec

Autostereoscopic

Holograms

Active stereoscopy

Passive Stereoscopy

Circular PolarizationLinear Polarization

Anaglyph

Infitec

- Eye separation by color spectrum -

Light Efficiency

http://www.barco.com/projection_systems/downloads/Stereoscopic_technologies_overview.pdfFriday, November 16, 12

Autostereoscopic

Holograms

Stores ALL light interactions through an entire “window”

Holograms

Stereoscopy for multiple users ?

Active + polarized

Sequential

6 users: stereo + tracking

6 usersstereoscopy + tracking

• video

Multi-display

What system can we create with only one graphics card?

What can we do with one graphics card?

Nvidia SLI - ATI CrossfireUse two graphics card for output on only one graphics cards

Alternate frame rendering

Split frame rendering

Scalable graphicsImage decomposition ( ~= SLI )Useful if application is fill-rate limited

frame rate decreases with increasing number of pixels (i.e. when the image resolution increases)

Scalable graphicsDatabase decompositionUseful if application is CPU limited

frame rate decreases with increasing size of 3D scenes

Scalable graphicsStereoscopy

How many displays with one computer ?

Why multi-display ?

Multiple display surfaces

Increase display size

Increase brightness

CPU/GPU Bottlenecks

Cluster

How ?Matrox DualHead2Go

Matrox TripleHead2Go

Multiple graphics card in a single PC

Often max 2 graphics cards

Multiple graphics cards in one PC

Multi-pipe ~= Multi-GPU

One rendering context per GPU

Problems of resources sharing, threading

Need specific graphics engine adaptations

But best performances

NVidia Quadro Plex•Up to :•4 GPU•4 outputs•16gb memory

•Can combine two Quadro Plex

Quadro Mosaic ModeMosaic mode hides multi-gpu

Only shows one gpu to system

Can have 4 outputs with 1 plex without needing multi-pipe adaptation

Good performances

ATI EyeFinity

Increase size or brightnessMersive autocalibration

What if need more displays ?

96 projectors ?

Cluster !

Cluster Synchronisation

Scene synchronisation (Scenelock)

Swap-Buffer synchronisation (Swaplock)

Eyes synchronisation (Genlock)

Cluster

Scalable

Each node can be cheap

Easy node replacement

Complex to manage/program

Tracking

TrackingMechanical

Fakespace’s Boom

Precise

Help Lift weight

Big, discontinued

TrackingMagnetic

Polhemus

Ascension

Precise, fast

Sensitive to metal

TrackingInertial

SpacePoint Fusion

XSens Mocap

Accelerometers, Gyroscopes, magnetometers

Very fast

Orientation onlyFriday, November 16, 12

TrackingOptical

ioTracker

OptiTrack

TrackIR

Wireless

No electronics

Install cameras

Line of sightFriday, November 16, 12

Leap motion

https://www.youtube.com/watch?v=N6hCwjwzUHgFriday, November 16, 12

TrackingMixing Optical

Intersense IS-900

Inertial + Ultrasound

IS-1200

Inertial + AR Patterns

Wiimote

Playstation Move

Optical Body Tracking

4D Views

http://www.4dviews.com/

Kinect fusion

http://www.youtube.com/watch?v=quGhaggn3cQ

Gloves

Haptics

HapticsForce feedback

TN Games - Gaming Vest

Locomotion

CirculaFloor

• http://www.youtube.com/watch?v=rYsvB2y2Ero

Virtusphere

• http://www.youtube.com/watch?v=qTnnJR-hS7k

OmniDirectionnal Treadmill

• http://www.youtube.com/watch?v=BQw1tsgrJOsFriday, November 16, 12

Kuka robot

http://www.cyberneum.de/f1/Friday, November 16, 12

Force Dynamics

http://www.force-dynamics.com/Friday, November 16, 12

Wizdish

http://www.wizdish.com

Applications

IVR Applications

Design, prototyping

Marketing

Training

Medical

Research

Geoscience, finance, archeologie, urbanism, sport, art...

iVR ApplicationsDesign, prototyping

Car, aircraft, boat, tractors, house appliances

Collaborative design review

Experience from user’s point of view

Easy and fast to prototype

More iterations, better products

(c) Peugeot PSA

(c) MieleFriday, November 16, 12

PSA Peugeot-Citroen - http://www.youtube.com/watch?v=WhMh191RltQ

http://cb.nowan.net/blog/2010/10/12/more-about-fords-vr-facilities/Friday, November 16, 12

http://vr4d.ingenierium.com/Friday, November 16, 12

Ergo-Wide

http://www.clarte.asso.fr/realite-virtuelle.php/en/ERGOWIDE%C2%B3/https://www.youtube.com/watch?v=aczLiNtVyDI

Lockheed Martin

http://www.youtube.com/watch?v=itOxIpj68MwFriday, November 16, 12

Mockup-builder

http://www.youtube.com/watch?v=ZxJD9DXDB1EFriday, November 16, 12

iVR ApplicationsMarketing

Product experience

Shopping experience

Procter&Gamble

Kimberley Clarke

(c) University of Gronigen

(c) Virtools

iVR ApplicationsTraining

Repeatability

Controlled environment

Can practice rare / dangerous situations

Can analyze / quantify performance

No health risk for patients

Easy accessibility

Reduced costs

Adaptable difficulty and assistance

Multiple users trained at same time

Attractive

(c) SNCF

(c) University of Florida

SNCF - Fiacre

SNCF - Simurat

http://www.youtube.com/watch?v=FEnalzHVmdI

Saint-Gobain & Clarte

Use less material

Environment friendly

No water, nothing to clean

Faster to setup

RE-lion

http://vimeo.com/25032061Friday, November 16, 12

ESCIN - Sivic - Crime scene investigation

http://www.youtube.com/watch?v=FC0gHv-Snmw

iVR ApplicationsResearch

Exposure therapy

PTSD, Phobias

Rehabilitation : motor, addictions

Create Empathy

Pain Distractionhttp://www.youtube.com/watch?v=SinlgK5Q4v4

Crescent - Acrophobia

Burn pain - Snow World

http://www.youtube.com/watch?v=jNIqyyypojgFriday, November 16, 12

Fear of flight

Arte - “A chacun sa réalité”

Summary

IVR strengths

Controlled / adaptable / safe environment

Natural interactions / reactions

Analyze

iVR Games ?

Cave Quake 3 (c) Paul Rajlich

Cave Unreal Tournament(c) Jeffrey Jacobson

https://www.youtube.com/watch?v=r580mzSxSp8

http://vimeo.com/9549613

Immersive rail shooter

VR Angry-bot

http://www.youtube.com/watch?v=QSXxjr1SnqAFriday, November 16, 12

Battlefield 3 simulator

www.youtube.com/watch?v=nQR49JGySTM

3D User Interactions

VR Techniques3D User Interactions

Bowman classification for generic interactionsSelection

Specifying one or more objects from a setManipulation

Modifying object properties (position, orientation, scale, shape, color, texture, behavior, etc.)Navigation

TravelWayfinding

System ControlMenusIssuing commandChanging modeChoosing tool

Symbolic InputOther interactions

Measuring… ? http://www.3dui.org

Navidget

http://www.youtube.com/watch?v=wpaKjLZll80Friday, November 16, 12

http://cb.nowan.net/blog/2010/03/25/3dui-grand-prize-2010-the-winners/

World builder

http://www.youtube.com/watch?v=VzFpg271sm8Friday, November 16, 12

Perceptive illusions

Body representation in VR

Proteus effect : virtual appearance can transform our behaviors towards other people ( Jeremy Bailenson )

It seems possible to transfer our body in virtual bodies

Rubber hand illusion

http://www.youtube.com/watch?v=TCQbygjG0RUFriday, November 16, 12

Body transfer

http://www.youtube.com/watch?v=8oF8sQvnTlMFriday, November 16, 12

Barbie doll illusion

http://www.youtube.com/watch?v=_z8pSTMfGSoFriday, November 16, 12

First person VR illusion

http://www.youtube.com/watch?v=DlbSVnVRLTUFriday, November 16, 12

Perceptive IlusionsPseudo haptics

http://www.irisa.fr/tactiles

http://kaji-lab.jp/ja/index.php?research

Perceptive IlusionsPseudo haptics

Perceptive IllusionsPlay with perception’s limits

Our perception is dominated by visual sense

Balance

Redirected Walking :

(c) Franck SteinickeFriday, November 16, 12

Change blindness :

(c) Evan Suma - USC/ICTFriday, November 16, 12

https://www.youtube.com/watch?v=FTWVjJ4um4A

Institute of creative technologies - change blindess

Perceptive IllusionsPassive haptics

(c) Luv KohliFriday, November 16, 12

Passive haptics

Ford PVM

Learn about human perception through its limitations

Improve VR experience by taking advantage of those limitations

Conclusion

It is possible to transfer your body in VR

Reality is not what you think

VR is just another reality

http://www.af-rv.frFriday, November 16, 12

http://www.youtube.com/watch?v=lHZltjDZBJo&feature=plcp

www.VRGeeks.orgFriday, November 16, 12

http://www.vrgeeks.org/vr-experiences/vrescapeFriday, November 16, 12

http://www.vrgeeks.org/vr-experiences/vrsadisticFriday, November 16, 12

http://www.vrgeeks.org/vr-experiences/insnaketionFriday, November 16, 12

VR ChallengesWhat we must solve

VR is mainly a SOFTWARE problem

“3DUI is a hard and unexplored software engineering (and ergonomic) problem”

Markerless tracking

Minimum latency

Lightweight

Wide FOV

Wireless

Full body haptics

Multi-sensorial experience

Locomotion

Avatars

VR For AllFriday, November 16, 12

VR “Experiences”

Your goal :

Create presence

And don’t break it !!!

Verdun 1916-Time machinehttp://www.youtube.com/watch?v=ZBfYIefenp0

Conclusion

Thank you !Questions ?

Sebastien.Kuntz@nowan.net

Immersive Virtual Reality

Documents

Transcript of Immersive Virtual Reality

Emotion Recognition in Immersive Virtual Reality: From ...

Designing Immersive Virtual Reality for Geometry Educationssemwal/CS677/GeomEducation01667626.pdf · Designing Immersive Virtual Reality for Geometry Education Hannes Kaufmann1 Institute

Auclair 3D, Immersive Virtual Reality Augmented Reality

Asch conformity experiment using immersive virtual reality

Using immersive virtual reality to enhance anatomical understanding

A virtual reality platform for immersive education in ...

Immersive Virtual Reality : Engineering Application … · Conclusion: Immersive virtual reality has shown potential in engineering application in a space environment however future

Immersive Media/ Virtual Reality Part 1

Immersive training systems: Virtual reality and education ...teilab-static.arch.tamu.edu/quek/Classes/Aware+EmbodiedInteractio… · Immersive training systems: Virtual reality and

Virtual Reality and Immersive Technologies › wp-content › uploads › 2019 › 10 › Virtual...Virtual Reality and Immersive Technologies: A Roadmap for New Forms of Audience

RabbitRun: An Immersive Virtual Reality Game for Promoting ...

Immersive 3D virtual reality imaging in planning minimally ...

Immersive Visualization Anatomical Environment using Virtual Reality Devices

Indoor Skydiving in Immersive Virtual Reality with Embedded … · Indoor Skydiving in Immersive Virtual Reality with Embedded Storytelling Horst Eidenberger, Annette Mossely Interactive

Immersive Environments and Virtual Reality: Systematic ... · with reality. Virtual reality and other immersive Information and Communication Technologies (ICT) have a high potential

Immersive Virtual Reality Physical Education Instructional ...

Designing Immersive Virtual Reality for Geometry Education

Immersive Virtual Reality & Learning

Immersive Virtual Reality and Gamiﬁcation Within ... · Immersive Virtual Reality and Gamiﬁcation Within Procedurally Generated Environments to Increase Motivation During Gait

Immersive Virtual Reality In e-mail: P.O.B.Holt@hw.ac.uk ...