1

z

deti departamento de electrónica telecomunicações e informática universidade de aveiro

Frameworks for Interactive

Virtual Environments

Paulo Dias

1

2

Outline

• Introduction

• VR Frameworks– Graphic engines

– Physics Engine

– 3D computer Graphics Software

– Game engines

– Virtual Environment framework

• pSIVE (Platform for Setting up Interactive Virtual Environments)– System Requeriments

– Architecture

– Development

– Updates

3

Introduction

• Creating a Virtual Environment

– Cost (Time & Money)

– Specific knowledge

• Several frameworks/tools

– Specific implementation

– Particular architecture

• How to ease?

– Tools for ‘laymen’

– Flexible yet simple

4

VR Frameworks

• Which framework is more suitable?

– How to interact/use the tool

– Documentation Availability

– Extensibility

– Which Modules/Interfaces it provides

– Users

Graphics engines

• OpenSceneGraph– open source C++ cross-platform toolkit for simulators, virtual reality and

games

– several NodeKits that fulfill a variety of purposes, such as particle

systems, text display, special effects, shadow , 3D interactive controls

,…

• OpenSG– real-time graphics programs, e.g. for virtual reality applications similar to

OpenSceneGraph

• VTK– Open Source library based in Object Oriented Programming

– multi-platform (Windows/Unix/Mac)

– C++ implementation with several wrappers for other languages (Tcl/Tk-

Python-Java)

• Many more …

5

Real-time physics engines

• Bullet– Free, open source

– Collision detection

– Soft and rigid body dynamics

• Newton Game Dynamics– Cross platform open source

– Simulates rigid bodies in games and other real-time application

• PhysX – multi-platform game physics

– support several devices (smartphones, GPU, CPU)

– Integrated in several game engines

– Acquired by Nvidia in 2008

– SDK provided to developers for free

• Many more…

6

2

3D Computer Graphics software

• Blender– free/open-source 3D computer graphics software used for creating

animated films, visual effects, art, 3D printed models, interactive 3D

applications and video games.

– include 3D modelling and several effects: texturing, raster graphics

editing, rigging and skinning, fluid and smoke simulation, particle

simulation, soft body simulation,…

– integrated game engine.

• OpenSpace3D– Free Open Source development platform for interactive real time 3D

projects

– User friendly, possible to create interactive 3D scene without writing any

code.

• 3D Studio Max– 3D Modeling, Animation & Rendering Software

– Autodesk, commercial

7

Game engines

• Unity– cross-platform game creation system that includes a game engine and

an integrated development environment.

– Graphic working environment that facilitates 3D scene creation and

layouts

– Physics engine that provides collision detection, forces, torque and

physic materials

– Two licensing options Unity and Unity Pro

• Unreal Engine 4– Multiplatform game development toolkit supports state of the art

graphical fidelity

– full access to the source code of the engine

– In 2015 released for free usage with a 5% commission on commercial

products.

– PhysX 3.3 physics engine for physical simulation

8

Virtual Environment Frameworks - not desktop

• inVRs

• VRJuggler

• Vizard

• 3DVIA

• others

9 10

inVRs

– Institute of Graphics and Parallel ProgrammingJohannes Kepler University, Linz, Austria (2006)

– C++

– OpenSG and OpenSceneGraph as a scene graph engine

– Network distributed virtual world

– Pre-defined navigation and interaction techniques

– Configurable via XML

– Mostly used on academic environmentsAustrian/German Universities

11

Source: http://www.invrs.org/app_gallery.php

Safety Training - SAVE

Walkthrough

Entertainment - Space Trash

inVRs Applications

12

VR Juggler

– Iowa Center for Emerging Manufacturing Technology, Iowa State University, Iowa, United States (1998)

– C++

– Supports many graphics Engines – OSG, OpenSG, OpenGL, VTK…

– Huge variety of modules

– Network distribution through NetJuggler

– Extended over time – Ex.: VR JuggLua

– Used for many different purposes – Commercial / Academic

• Mina do Lousal (PT)

• Korea Ocean Research and Development Institute

– Configurable via VRJConf – Backend XML

3

13

Source:

Korea Ocean Research and Development Institute

Source: http://vrjuggler.org/, Last

Access Dec 03 2013

VR Juggler

14

Vizard

Vizard Development Edition

Vizard Development Academic - Single License $ 3,990

Vizard Development Academic - SITE License 5 Seats $ 6,990

Vizard Enterprise Edition

Vizard Enterprise Academic - Single License $ 5,990

Vizard Enterprise Academic - SITE License 5 Seats $ 9,990

Vizard Plug-Ins

Vizard ARToolworks Add-on Development Edition $ 570

Vizard ARToolworks Add-on Enterprise Edition $ 7,990

• WorldViz, Santa Barbara - CA, United States

• Pyton Scripts + GUI

• OpenSceneGraph - Possible to expand vizard funcionalities

• Support many comercial Devices

• Physics Engine

• Research & Science (VHIL - Stanford U), Architecture & Construction (Archidimex - Netherlands)

15

Vizard

Development EnvironmentApplications

• Dassault Systèmes, Vélizy-

Villacoublay, FRANCE

• Former Virtools

• Scripting Language + LUA

• Closed Engine

• Many GUI Tools to work on models,

animations and scenes

• Price around 9000 USD + 3000 USD/yr

Source:

http://www.3ds.com/fileadmin/PRODUCTS/3

DVIA/3DVIAVirtools/demoshowcase/html/bro

wse.html?br=1

3DVIA Studio

16

Peripherals Interfaces

• Virtual Reality Periphery Network (VRPN)– network-transparent interface between applications and physical

devices used in VR

– client-server approach

– layer of abstraction that classifies inputs into several categories (Analog,

Button, Dial, Force Device, Sound,Text, and Tracker), allowing to

receive generic input from different devices.

17

Other Frameworks

• FreeVR

• Avango NG

• DIVERSE

• ViRAL

• EON Studio

• …

18

4

Conclusion

• Difficult choice, many solutions

• Complex Architecture – Many Abstractions

• Often not trivial to Install / Configure

• Game engine emerging as alternative to

expensive frameworks

19

pSIVE – Danilo Souza 2013/2014

• Platform for Setting up Interactive

Virtual Environments (pSIVE) 2013/2014

– Virtual Visits

– Training

– Marketing

• Easy Creation

• Abstraction Layer

– Users – Frameworks

20

System Requeriments

• Hardware Independent

• Simple Configuration

– 3D Model Loading

– Attatch/Display Multimedia Information

– Technical aspects

– Reusable

• Well Defined Art Pipeline

– CAD softwares -> pSIVE

– Avoid calculations

• Cover part of IEETA’s devices

21 22

Realidade Virtual - equipamento

• Tracker 3DOF (Yaw, Pitch,

Roll) wired and wireless

• (2) stereoscopic

svga HMD

• WiiMote

• Wintracker 6DOF

• Kinect v1 and v2

• (2) Oculus Rift

• Razor Hydra

• Kinect

• Phantom

Architecture

• VR Juggler– Qualities met project requeriments

– Active Community and Develop

– Well documented

• OpenSceneGraph (OSG)– Easy-to-work-with solution

– Books available

– Very active project

– Support vast list of formats/software• 3D Studio Max, SketchUp, Maya, Blender...

• Why not Game Engines?

23

Development

• Create Base Layout

• Place and Adjust Elements

• Export Elements individually to OSG Format

Modeling Tool

• Choose Devices

• Load 3D Elements

• Attatch Information to elements

• Select Interaction and Manipulation Techniques

• Export Layout

ConfigInterface

• Load Devices

• Load Layout

• Parse Information

• Run Environment

Virtual Environment

24

5

Platform Overview

25

Multimedia Module

26

Interaction Module

27

Menu Module

28

Case of Study

PRODUTECH-PTI / Tegopi

29

Videos

30

6

Update pSIVE – João Pinto 2014/2015

• Support for newer HMD’s (oculus rift)

– OpenScenGraph with osgOculusViewer

• Kinect support

– VRPN and FAAST

João Pinto 2014/2015

31

Architecture

32

33

Mobile Interaction – Luis Afonso 2015-2016

• Move to Unity

– Oculus, leap and Android support

34

Architecture

35 36

7

Bibliography

• Burdea, G. C., & Coiffet, P. (2003). Virtual Reality Technology (2nd ed.). New York,

NY, USA: John Wiley & Sons, Inc.

• Bowman, D. A., Johnson, D. B., & Hodges, L. F. (1999). Testbed evaluation of

virtual environment interaction techniques. In Proceedings of the ACM symposium

on Virtual reality software and technology - VRST ’99 (pp. 26–33). New York, New

York, USA: ACM Press.

• Craig, A., Sherman, W. R., & Will, J. D. (2009). Developing Virtual Reality

Applications: Foundations of Effective Design. San Francisco, CA, USA: Morgan

Kaufmann Publishers Inc.

• Gutiérrez, M. A. A., Vexo, F., & Thalmann, D. (2008). Stepping into Virtual Reality

(p. 214). London: Springer London.

• Ii, Russell M Taylor et al. (2001). “VRPN : A Device-Independent , Network-

Transparent VR Peripheral System”. In: VRST, pp. 55–61.

• Suma, Evan A. et al. (2011). “FAAST: The flexible action and articulated skeleton

toolkit”. In: Proceedings - IEEE Virtual Reality, pp. 247–248. isbn: 9781457700361

• Bierbaum, Allen et al. (2001). “VR Juggler: A Virtual Platform for Virtual Reality

Application Development”. In: Proceedings of the Virtual Reality 2001 Conference

(VR’01). VR ’01.Washington, DC, USA: IEEE Computer Society

37