Virtual and Augmented Reality – 2016

MIECT, MIEET, MEI

Introduction to course

Universidade de Aveiro

Departamento de Electrónica,

Telecomunicações e Informática

Realidade Virtual e Aumentada Beatriz Sousa Santos, 2016/2017

2

Virtual and Augmented (Mixed) Reality

• Has been expanding from a research field into a commercially viable

technology.

• is highly interdisciplinary, combining work in diverse disciplines:

- more technology oriented research:

computer graphics, display technology, computer vision,

sensors, signal processing and computer networking.

and

- human centered research:

interaction techniques, design, human factors,

wearable computing and mobile computing.

• The growing interest in VAR applications is creating new challenges

for research in all of these areas

3

What is Virtual Reality?

“For better or worse, the label virtual reality stuck to this particular branch of computer graphics.

I define a virtual reality experience as any in which the user is effectively immersed in a responsive virtual world. This implies user dynamic control of viewpoint.”

“A high-end user-computer interface that involves real-time simulation and interaction through multiple sensorial channels (vision, sound, touch, smell, taste)”.

(Fred Brooks, 1999)

4

(Burdea et al., 2004)

Virtual Reality in practice for years – examples

Industry

Jaguar / Land Rover

Ford

http://www.youtube.com/watch?v=j3qcnvgVlNk

https://www.youtube.com/watch?v=BEFfp2QhHZ

U

5

http://www.youtube.com/watch?v=CMU0HtmBt48

6

Medicine

http://www.youtube.com/

watch?v=zJmrcEM-uvA

7

Entertainment

http://www.oculusvr.com/

Oculus Rift

2014; ~300 USD

8

http://www.youtube.com/watch?v=N8uuDT5AYts

“AR is a novel human machine interaction tool that overlays computer-

generated information on the real world environment. The information display

and image overlay are context-sensitive, which means that they depend on the

observed objects.” (Ong et al., 2008)

10

What is Augmented Reality?

• According to Azuma Augmented Reality systems have the following

characteristics:

“1) Combine real and virtual

2) Interactive in real time

3) Registered in 3D”

11

http://www.youtube.com/watch?v=8UxWkZtUKaI

Early Augmented Reality – examples

Lego

12

Ray Ban Virtual Mirror

13

http://www.youtube.com/watch?v=Ag7H4YScqZs

http://www.ray-

ban.com/usa/virtual-mirror

Augmented Reality Sandbox

14

http://idav.ucdavis.edu/~okre

ylos/ResDev/SARndbox/

http://www.techradar.com/reviews/wearables/microsoft-hololens-1281834/review

The Future of AR?

Hololens

2016; ~3000 USD?

https://www.microsoft.com/microsoft-hololens/en-us

R. Furlan,”The future of augmented reality: Hololens - Microsoft's AR headset

shines despite rough edges”, IEEE Spectrum, 2016, 53 (6), p. 21

Mixed Reality: “Continuum”

“Augmenting natural feedback to the operator with simulated cues”

16

(Milgram et al., 1994)

Mixed Reality (MR)

Real Envinoment Augmented Reality Augmented Virtuality Virtual Reality

(Steinicke et a., 2009) http://www.youtube.com/watch?feature=endscreen&v=UgQfo7eNFdw&NR=1

Team

Beatriz Sousa Santos + Paulo Dias

Other courses by this team:

Human-Computer Interaction, Visual Computing, Information

Visualization, Computer Graphics, 3D Modeling and Visualization

Interactive Computer Systems

Dissertations:

Medical and Scientific Data/Information Visualization, Virtual Reality,

Augmented Reality, Interaction with large displays …

18

We thank all the people involved:

• Guilherme Campos, Samuel Silva, Carlos Ferreira

• MSc and PhD Students:

Mário Cruz, Paulo Moreira; Angela Pimentel; Gustavo Corrente; Ana

Cantanhede; Jan-Willem Baggerman, Pauline van der Horst, Frank van

Huussen; Ricardo Casaleiro; Ricardo Seco; Paulo Nascimento, Mauro

Ribeiro; Bruno Morais; Nuno Veloso; Danilo Sousa, João Pinto, Sérgio

Eliseu, João Cardoso, João Pinto, Luís Afonso ...

• And all participants in demos, tests and usability studies...

20

Topics

• Introduction to Virtual Reality (VR) and Augmented Reality (AR)

Definition, historical perspective and evolution, examples

• Modeling of Virtual Environment (VE)

• Input and output devices, tracking, cameras

• 3D user interfaces and interaction techniques

• Human factors in VR and AR

• Frameworks for VR and AR

• Traditional and emerging applications

http://sweet.ua.pt/bss

21

Lectures and Lab classes (tentative)

1 (13/09/16) - Introduction to the course

2 (20/09/16) - Introduction to Virtual and Augmented Reality

Paper presentation Guidelines

Frameworks para RV e RA / VR and AR Frameworks

3 (27/9/16) - Introduction to Virtual and Augmented Reality (cont.)

Unity

4 (4/10/16) - Input Devices / Paper presentation

Unity

5 (11/10/16) – Visit to Centro Rovisco Pais (to be confirmed)

6 (18/10/16) - Output Devices (visual) - Stereoscopy / Paper presentation

Unity - Practical assignment

22

7 (25/10/16) – Unity - Practical assignment

8 (8/11/16) - Output Devices (other) - Paper presentation

Unity – Practical assignment

9 (15/11/16) - Interaction in VR and AR - Paper presentation

Introduction OpenCV

10 (22/11/16) – Cameras and world registration - Paper presentation

Camera calibration with OpenCV

10 (29/11/16) - Human Factors in VR and AR - Paper presentation

Toolkits for AR (ARToolkit and others)

12 (06/12/16) – Hardware Demo

13 (13/12/16) – Assignments: demo and presentation

14 (20/12/16) - Physics modelling/ Paper presentation

23

Lectures and Lab classes

• ~1h30 min – lecture; including paper presentation and discussion

• ~1h 30 min – lab

• 2 sessions devoted to presentation and demo of the assignments

Unity, OpenCV, ARToolkit ....

24

Main bibliography

• Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and

Morgan & Claypool, 2016

• Schmalstieg, D., Hollerer, T., Augmented Reality: Principles and Practice (Usability).

Addison-Wesley Professional. (2016).

• Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications:

Foundations of Effective Design, Morgan Kaufmann, 2009

• Burdea, G., P. Coiffet, Virtual Reality Technology, 2nd ed., Wiley, 2003

• Bowman, D., E. Kruijff, I. Poupyrev, J. LaViola, 3D User Interfaces: Theory

and Practice, Addison Wesley, 2004

• Anthes, C., M. Wiedemann, D. Kranzlmüller, “State of the Art of Virtual Reality

Technology”, IEEE Aerospace Conference, 2016.

• Journal and Conference Papers

25

Other bibliography

• Kim, G., Designing Virtual Reality Systems: The structured approach, Springer 2005

• Mullen, T., Prototyping Augmented Reality, Sybex, 2011

• Furht (ed.), Handbook of Augmented Reality, Springer, 2011

• Stanney, K.(ed.), Handbook of Virtual Environments, Design, Implementation and Applications, Lawrence Erlbaum Associates, 2002

• Journal and Conference Papers

26

- Exam – 40%

- Programming assignment – 20% + 30%

- Paper presentation and discussion – 10%

• Working students must decide on the type of assessment and inform us until September, 30

Assessment

27

Practical assignment (general characteristics)

• Groups of two students

• Mini-project

• Development of a simple application using specific devices and

libraries

• Existing code may be used, provided that it is explicitly identified

and its source adequately cited

29

This year’s “client”:

30 http://www.tsf.pt/portugal/saude/interior/hospital-rovisco-pais-e-central-na-recuperacao-de-

doentes-com-avc-4486208.html

Our devices

31

Analyze and present a conference/ journal paper:

1 student – short paper (VR2016, ISMAR2015)

2 students– long paper (SIGGRAPH2016 or journal)

Each student must:

- Decide if he/she will present alone or with a colleague

- select two papers

or send preferences to (bss@ua.pt)

until September, 29

- Propose another paper

- Read the paper presentation guidelines (presentation guidelines)

- Make a 12 mim (short paper) or a 25 min (2 students - long paper) presentation

- Send the slides to (bss@ua.pt)

33

Papers to read, present and discuss

http://ismar.vgtc.org/

http://ieeevr.org/2015/

34

http://ieeexplore.ieee.org/Xplore/

http://dl.acm.org/

http://s2016.siggraph.org/

http://www.springer.com

/computer/image+proce

ssing/journal/10055

Discuss papers

• After each paper presentation:

- All students vote on their colleagues presentation

(through googleforms)

- Two students will discuss the paper

35

https://docs.google.com/forms/d/1rXAb2LlJTaj74-

xWMhhYMwWcNoUlzBUiyY9gZp2OFT0/viewform

This year’s “client”:

36 http://www.tsf.pt/portugal/saude/interior/hospital-rovisco-pais-e-central-na-recuperacao-de-

doentes-com-avc-4486208.html

• Students’ profile?

• Questions?

37

Interesting recent papers and links

• Rod Furlan,”The future of augmented reality: Hololens - Microsoft's

AR headset shines despite rough edges”, IEEE Spectrum, 53 (6), p.

21, DOI: 10.1109/MSPEC.2016.7473143

38