ByIvan Trajkovic

andDamir Burazerovic

AUGMENTED REALITY AND 3D

INDEX

• What is Augmented Reality ?

• Problems in Augmented Reality

• Augmented Reality on Android platform

• Development platform and system requirements

• LG Optimus 3D additional requirements

• ARToolKit

WHAT IS AUGMENTED REALITY (AR) ?

• It is a variation of Virtual Environments (VE)• It is a system that supplements the real world with

computer-generated virtual objects which seem to coexist in the same space and present the following properties:– It combines real and virtual objects in real environment– It executes interactively in real time– It lines up real and virtual objects– It is applied to all senses of the user

PROBLEMS IN AUGMENTED REALITY

There are three major challenges associated with AR:

CAMERA TRACKING

HANDLING OCCLUSIONS

ILLUMINATION CONSISTENCY

CAMERA TRACKING

Fig 1: the Simple Program Running a virtual block aligned

with a real mark

Fig 2: Threshold Video Image with Identified Pattern

HANDLING OCCLUSIONS

Problem: real objects are in front of the virtual objects in the scene

ILLUMINATION CONSISTENCY

Three frames from a 3 hour long sequence showing virtual sculpture rendered into scene with consistent illumination.

AUGMENTED REALITY ON ANDROID PLATFORM

• AndAR - Android Augmented Reality.– ARToolKit for Android.

( http://code.google.com/p/andar/ )

• NyARToolkit.- Java/C#/Android.

( http://sourceforge.jp/projects/nyartoolkit/ )

DEVELOPMENT PLATFORM AND SYSTEM REQUIREMENTS

• Multiplatform (Linux, MacOS and Windows)– Windows XP (32-bit), Vista (32- or 64-bit), or Windows 7 (32- or 64-

bit)– Mac OS X 10.5.8 or later (x86 only)– Linux (tested on Ubuntu Linux, Lucid Lynx)

• Android SDK– (http://developer.android.com/sdk/index.html)

• Eclipse IDE

• ADT Plugin for Eclipse

LG OPTIMUS 3D ADDITIONAL REQUIREMENTS

• Real3D SDK – ( http://developer.lgmobile.com/sdk/android/repository.xml )

• What LG Real3D provides– Real3D API that helps you to control the display hardware block

interliving– Real3D API demos– The emulator that enables you to test your 3D applications with red-

cyan glasses

WHAT IS Real3D?

HOW Real3D WORKS?

Source: LG Real3D SDK

ARToolKit

• Software library for building AR applications.

• Originally developed by Dr. Hirokazu Kato.– Human Interface Technology Laboratory (HIT Lab) at the University of

Washington,– HIT Lab NZ at the University of Canterbury, New Zealand,– ARToolworks, Inc, Seattle.

ARToolKit

• Uses computer vision algorithms to solve AR tracking problem.

• Calculate the real camera position and orientation relative to physical markers in real time.

ARToolKit FEATURES

• Multiplatform (Linux, MacOS and Windows)

• Single camera tracking using any black square.

• Distributed with complete source code.

• Fast rendering based on OpenGL.

• Simple and modular API (in C and C++).

• A complete set of samples and utilities.

ARToolKit FEATURES

• OpenSource with GPL license for non- commercial usage.

• Commercial licenses are administered by ARToolworks, Inc., Seattle, WA, USA.

ARToolKit ARCHITECTURE

ARToolKit has three basic modules:

1. Scene Capturer

2. Augmented Reality (AR)

3. Augmented Image Generator

HOW DOES ARToolKit WORK?

Source: http://www.hitl.washington.edu/artoolkit/documentation/userarwork.htm

LIMITATIONS

• Virtual objects appear only when complete markers are visible.

• Size of virtual objects

• Movement of virtual objects

• If users cover up part of the pattern with their hands or other objects the virtual object will disappear

LIMITATIONS

• Range issues

• Pattern complexity

• Marker orientation relative to the camera

• Lighting conditions

ARToolKit – DEVELOPMENT PRINCIPLES

There are two stages when developing an ARToolKit application:

1. Writing the application.

2. Training image-processing routines on markers that will be used in the application.

ARToolKit – Development Principles

Initialization 1. Initialize the video capture and read in the marker pattern files and camera parameters.init()

Main Loop 2. Grab a video input frame. arVideoGetImage()

3. Detect the markers and recognized patterns in the video input frame.arDetectMarker()

4. Calculate the camera transformation relative to the detected patterns.arGetTransMat()

5. Draw the virtual objects on the detected patterns.draw()

Shutdown 6. Close the video capture down.cleanup()

LITERATURE

• Augment Reality (Edited by SohaMaad), 2010 Intech

• ARToolKithttp://www.tinmith.net/lca2004/ARToolkit/ARToolKit2.33doc.pdf

• Real-Time Occlusion Handling in Augmented Reality Based on an Object Tracking Approach

Authors: Yuan Tian, Tao Guan* and Cheng Wang http://www.mdpi.com/1424-8220/10/4/2885/pdf

LINKS

• ARToolKithttp://www.hitl.washington.edu/artoolkit/

• ARToolkitPro for Androidhttp://www.artoolworks.com/support/library/

ARToolKit_Professional• ARToolkitPlus 2.1.1

http://handheldar.icg.tugraz.at/artoolkitplus.php• NyARToolkit for Android

http://sourceforge.jp/projects/nyartoolkit/releases/?package_id=7605