L 3 AR D l T lLecture 3: AR Developer Tools

Mark Billinghurst

mark.billinghurst@hitlabnz.org

July 2011

COSC 426: Augmented RealityCOSC 426: Augmented Reality

Building Compelling AR ExperiencesBuilding Compelling AR Experiences

experiences

applications

toolstools

components Display, Tracking

Sony CSL © 2004

Low Level AR Libraries

ARToolKit Marker based trackingg

FLARToolKitFl h f ART lKFlash version of ARToolKit

SSTTSimple Spatial Template Tracking

O iraOpiraRobust Natural Feature Tracking

Wh ART lK ?What is ARToolKit?

Marker Tracking Library for AR applicationsOpen Source Multi platform (Linux Windows MacOS)Open Source, Multi-platform (Linux, Windows, MacOS)

Overlays 3D virtual objects on real markersUses single tracking marker Determines camera pose information (6 DOF)

ARToolKit Websitehttp://www.hitl.washington.edu/artoolkit/http://artoolkit.sourceforge.net/

ART lK S fARToolKit SoftwareARToolKit version: 2 65 or laterARToolKit version: 2.65 or laterCurrently two license models

Open Source (GPL): ARToolKit 2 72Open Source (GPL): ARToolKit 2.72Commercial (ARToolWorks): ARToolKit 4.0

OS Li Wi d M OS X iPh /A d idOS: Linux, Windows, MacOS X, iPhone/AndroidProgramming language: CR l d fRelated software

ARToolKit Professional: Commercial versionARToolKitPlus: Advanced versionARToolKitPlus: Advanced versionNyARToolkit: Java and C# versionFLARToolKit: Flash version

ART lKit F ilARToolKit Family

ARToolKitARToolKit NFT

ARToolKit Plus

JARToolKit (Java)

ARToolKit (Symbian)

FLARToolKit (Flash)NyToolKit- Java, C#,- Android WM

FLARToolKit (Flash)

( )Android, WM FLARManager (Flash)

ART lK ARToolKit contents

LibrarieslibAR – trackingb t ac glibARvideo – video capturinglibARgsub – image/graphics drawinglibARgsub image/graphics drawinglibARmulti – multi-marker tracking

UtilitiesUtilitiesCamera calibrationMarker training

ARToolKit Structure

ARvideo.lib

DirectShow

Three key libraries:AR32 lib ARToolKit image processing functionsAR32.lib – ARToolKit image processing functionsARgsub32.lib – ARToolKit graphics functionsARvideo lib – DirectShow video capture classARvideo.lib – DirectShow video capture class

Additional Software

To build an AR application you may need additional softwareadditional softwareHigh level rendering library

Open VRML, Open Inventor, osgART, etc

Audio LibraryAudio LibraryFmod, etc

Peripheral support

What does ARToolKit Calculate?Position of makers in the camera coordinatesPose of markers in the camera coordinatesPose of markers in the camera coordinatesOutput format

3x4 matrix format to represent the transformation matrix from the marker coordinates to the camera coordinates

C d SCoordinate Systems

Tracking Range with Pattern Size

Rule of thumb – range = 10 x pattern widthRule of thumb range 10 x pattern width

Tracking Error with Range

Tracking Error with Angle

AR Application Development

An ARToolKit ApplicationI iti li tiInitialization

Load camera and pattern parametersM i LMain Loop

Step1. Image capture and displayStep2. Marker detectionStep3. Marker identificationStep4. Getting pose informationStep5. Object Interactions/SimulationStep6. Display virtual objects

End ApplicationCamera shut down

Image capture: libARvideo

Return the pointer for captured imageARUint8 *arVideoGetImage( void );

Pixel format and byte size are defined in config hPixel format and byte size are defined in config.h#define AR_PIX_FORMAT_BGR

#define AR_PIX_SIZE 3

Graphics handling: libARgsub

Set up and clean up the graphics windowvoid argInit( ARParam *cparam double zoomvoid argInit( ARParam *cparam, double zoom,

int fullFlag, int xwin, int ywin,int hmd_flag );

void argCleanup( void );

cparam: camera parameter

zoom: zoom ratiofullFlag: 0: normal, 1: full screen modeXwin, ywin: create small window for debughmd_flag: 0: normal, 1: optical see-through mode

Graphics handling: libARgsubGo into the iterative cyclevoid argMainLoop(

void (*mouseFunc)(int btn int state int x int y)void (*mouseFunc)(int btn,int state,int x,int y),void (*keyFunc)(unsigned char key, int x, int y),void (*mainFunc)(void)

));

Swap bufferspvoid argSwapBuffers( void );

Graphics handling: libARgsubSet the window for 2D drawing

id D M d 2D( id )void argDrawMode2D( void );

Set the window for 3D drawinggvoid argDrawMode3D( void );

void argDraw3dCamera( int xwin, int ywin );

Display imageid Di I ( ARUi t8 *ivoid argDispImage( ARUint8 *image,

int xwin, int ywin );

Sample ARToolKit Applications

Ex. 1: Simple video displayEx. 2: Detecting a markerEx. 3: Using patternE 4 G tti 3D i f tiEx. 4: Getting a 3D informationEx 5: Virtual object overlayEx. 5: Virtual object overlay

Ex 1: Simple Video Display

Program : sample1.cKey points

Loop structureLoop structureVideo image handlingCamera parameter handlingWindow setupWindow setupMouse and keyboard handling

Sample1.c Main Functionmain(){{

init();argMainLoop( mouseEvent, keyEvent,

mainLoop);a oop);}

Sample1.c - mainLoop Functionif( dataPtr = (ARUint8 *)arVideoGetImage()) == NULL ) {arVideoGetImage()) NULL ) {

arUtilSleep(2); return;return;

}D M d 2D()argDrawMode2D();

argDispImage(dataPtr, 0, 0 ); arVideoCapNext(); argSwapBuffers();

Sample1.c – video initializationC fi th id i tConfigure the video inputvconf = <video configuration string>

Start video capturearVideoCapStart();

In init(), open the videoarVideoOpen( vconf );arVideoInqSize(&xsize, &ysize);

When finished, close the video pathWhen finished, close the video patharVideoCapStop();arVideoClose();

Changing Image Size

For input capturevconf = “videoWidth=320,videoHeight=240";

Note – the camera must support this image size

For displayargInit( &cparam 1 5 0 0 0 0 );argInit( &cparam, 1.5, 0, 0, 0, 0 );

Th d f d l The second parameter means zoom ratio for display image size related to input image.

Ex. 2: Detecting a MarkerP l 2Program : sample2.cKey pointsy p

Threshold valueImportant external variablesImportant external variablesarDebug – keep thresholded imagearImage – pointer for thresholded imagearImage – pointer for thresholded imagearImageProcMode – use 50% image for image processingprocessing

- AR_IMAGE_PROC_IN_FULL - AR IMAGE PROC IN HALFAR_IMAGE_PROC_IN_HALF

Sample2.c – marker detection

/* detect the markers in the video frame */ if( D t tM k (d t Pt th hif(arDetectMarker(dataPtr, thresh, &marker_info, &marker_num) < 0 ) {l ()cleanup();

exit(0);}for( i = 0; i < marker_num; i++ ) {argDrawSquare(marker_info[i].vertex,0,0);

}

Sample2.c – marker_info structuretypedef struct {

int area; int id; int dir; double cf; double pos[2]; double line[4][3]; double vertex[4][2];

} ARMarkerInfo;

Ex. 3: Using a Pattern

Program : sample3.cKey pointsKey points

Pattern files loadingStructure of marker information

- Region features- Pattern Id, direction- Certainty factor

Marker identificationMarker identification

Making a pattern templateU f ili Use of utility program:

mk_patt.exeShow the patternShow the patternPut the corner of red line segments on the left-top segments on the left top vertex of the markerPattern stored as a template in a file1:2:1 ratio determines the

tt i dpattern region used

Sample3.c – Pattern File Loading

int patt_id;char *patt name = “Data/kanjiPatt”char patt_name Data/kanjiPatt

/* load pattern file *// load pattern file /if(patt_id=arLoadPatt (patt_name) < 0){{

printf ("Pattern file load error !! \n"); exit(0);exit(0);

}

Checking for known patterns/* check for known patterns */k = -1;for( i = 0; i < marker num; i++ ) {for( i = 0; i < marker_num; i++ ) {

if( marker_info[i].id == patt_id) {/* you've found a pattern */printf("Found pattern: %d \n",patt_id);if( k == -1 ) k = i;elseelse /* make sure you have the best pattern (highest confidence factor) */if( k i f [k] f k i f [i] f )if( marker_info[k].cf < marker_info[i].cf )

k = i;}

}

Ex. 4 – Getting 3D information

Program : sample4.cKey points

Definition of a real markerTransformation matrixTransformation matrix- Rotation component

T l ti t- Translation component

Sample4.c – Transformation matrix

double marker_center[2] = {0.0, 0.0};double marker_width = 80.0;double marker trans[3][4];_

arGetTransMat(&marker info[i]arGetTransMat(&marker_info[i], marker_center, marker_width, marker trans);marker_trans);

Finding the Camera PositionThis function sets transformation matrix from marker

to camera into marker_trans[3][4]._ [ ][ ]arGetTransMat(&marker_info[k], marker_center,

marker_width, marker_trans);

You can see the position information in the values of marker trans[3][4]marker_trans[3][4].

Xpos = marker_trans[0][3];Ypos = marker_trans[1][3];Zpos = marker_trans[2][3];

ARToolKit Coordinate Frame

Ex. 5- Virtual Object Display

Program : sample5.cKey points

OpenGL parameter settingOpenGL parameter settingSetup of projection matrixSetup of modelview matrix

Appending your own OpenGL code

Set the camera parameters to OpenGL Projection matrix.argDrawMode3D();gargDraw3dCamera( 0, 0 );

Set the transformation matrix from the marker to the camera to Set the transformation matrix from the marker to the camera to the OpenGL ModelView matrix.argConvGlpara(marker_trans, gl_para);glMatrixMode(GL_MODELVIEW);glLoadMatrixd( gl_para );

After calling these functions, your OpenGL objects are drawn in the real marker coordinates.

3D CG Model RenderingARToolKit does not have a function to handle 3D CG models3D CG models.3rd party CG rendering software should be employed.

OpenVRMLpOpenSceneGraphetcetc

Loading Multiple Patterns

Sample File: LoadMulti.cUses object c to load Uses object.c to load

Object Structuret d f t t {typedef struct { char name[256]; int id; int visible;double marker_coord[4][2];double trans[3][4];double trans[3][4];double marker_width;double marker_center[2];

} ObjectData_T;

Finding Multiple Transforms

Create object listObjectData T *object;ObjectData_T object;

Read in objects - in init( )read_ObjData( char *name, int *objectnum );

Find Transform in mainLoop( )Find Transform – in mainLoop( )for( i = 0; i < objectnum; i++ ) {

Ch k tt..Check patterns..Find transforms for each marker}}

Drawing Multiple Objects

Send the object list to the draw functiondraw( object, objectnum );

Draw each object individuallyj yfor( i = 0; i < objectnum; i++ ) {

if( object[i].visible == 0 ) continue;( j [ ] ) ;argConvGlpara(object[i].trans, gl_para);draw object( object[i].id, gl para);draw_object( object[i].id, gl_para);

}

Limitations of ARToolKitPartial occlusions cause tracking failureAffected by lighting and shadowsAffected by lighting and shadowsTracking range depends on marker sizeP f d d b f kPerformance depends on number of markers

cf artTag, ARToolKitPlusPose accuracy depends on distance to markerPose accuracy depends on angle to markery p g

ARToolKit in the World

H d d f jHundreds of projectsLarge research communityg y

FLARToolKitFlash AS3 Version of the ARToolKit (was ported from NyARToolkit the Java Version of the ARToolkit)

enables augmented reality on the Browseruses Papervision3D for as 3D Engineavailable at http://saqoosha.net/dual license, GPL and commercial license

AR Application Components

Adobe Flash

Papervision 3D

FLARToolkit

Source PackagesO i i l“ FLART lki„Original“ FLARToolkit (Libspark, Saqoosha)

(http://www.libspark.org/svn/as3/FLARToolKit/trunk/ )

Start-up-guidesSaqoosha (http://saqoosha.net/en/flartoolkit/start-up-guide/ )

Miko Haapoja (http://www.mikkoh.com/blog/?p=182 )

Frameworks“„FrameworksSquidder MultipleMarker – Example (http://www.squidder.com/2009/03/06/flar-how-to-multiple-instances-of-multiple-

k /markers/ )

FLARManager (http://words.transmote.com/wp/flarmanager/ )

Papervision 3Dhttp://www.papervision3d.org/Flash-based 3D-EngineFlash based 3D EngineSupports

import of 3D Modelstexturingganimationscene graphscene graph

alternatives: Away3d, Sandy,…

Papervision ExamplesRed Bull|Flugtage Flight Labhttp://www.redbull.com/flightlab/#/Game/TEMP_0

Barcinski JeanJean | 3D PortfolioBarcinski JeanJean | 3D Portfoliohttp://www.barcinski-jeanjean.com/

more cool papervision websiteshttp://blog.papervision3d.org/2009/04/24/2009-webby-awards-nominees/

AR Tools

B ildi C lli AR E iBuilding Compelling AR Experiences

experiences

applications

tools Authoringtools Authoring

components Tracking, Display

Sony CSL © 2004

AR AuthoringS ft Lib iSoftware Libraries

osgART, Studierstube, MXRToolKitgPlug-ins to existing software

DART (M di Di ) ARDART (Macromedia Director), mARxStand Alone

AMIRE, ComposAR, etcNext Generation

iaTAR (Tangible AR)iaTAR (Tangible AR)

mARx Plug-in

3D Studio Max Plug-inCan model and view AR content at the same time

BuildAR

http://www.buildar.co.nz/Stand alone applicationppVisual interface for AR model viewing application Enables non-programmers to build AR scenesEnables non-programmers to build AR scenes

ImageTclARAdds AR components to ImageTcl

http://metlab.cse.msu.edu/imagetclar/Modular Library (Scripting, Tcl)Supports several tracking systems (vision, magnetic, inertial)Easy to learn but little support, small community

DARTDesigners AR Toolkit

http://www cc gatech edu/dart/http://www.cc.gatech.edu/dart/AR plug-in for Macromedia DirectorD l d f d iDeveloped for designersVisual programmingScripting interface

StudierstubeComplete authoring toolComplete authoring tool

http://studierstube.icg.tu-graz.ac.at/Framework (Low Level Programming, C++)Modularity, Extensibility, Scalability, HeterogeneityModularity, Extensibility, Scalability, HeterogeneitySupport for wide range of trackers, displays, input

Metaio UnifEye SDKComplete commercial authoring platform

http://www.metaio.com/products/p pOffers viewer and editor toolsVisual interface and low level SDKVisual interface and low level SDKDelivery on desktop or mobile platforms

OSGART Programming LibraryIntegration of ARToolKit with a High-Level Rendering Engine (OpenSceneGraph)g g ( p p )OSGART= OpenSceneGraph + ARToolKit

Supporting Geometric + Photometric Registrationpp g g

osgART:Features

C++ (but also Python Lua etc)C++ (but also Python, Lua, etc).Multiple Video Input supports:

Direct (Firewire/USB Camera), Files, Network by ARvideo, PtGrey, CVCam, VideoWrapper, etc.

Benefits of Open Scene Graph Rendering Engine Plug-ins etcRendering Engine, Plug ins, etc

More Libraries

JARToolKitMRToolKit MXRToolKit ARLib OpenVIDIAMRToolKit, MXRToolKit, ARLib, OpenVIDIADWARF, Goblin XNAAMIRED’F iD’Fusion

Advanced Authoring: iaTAR (Lee 2004)

I i AR A th iImmersive AR AuthoringUsing real objects to create AR applications

osgART

Developing Augmented Reality Developing Augmented Reality Applications with osgART

What is a Scene Graph?Tree-like structure for organising a virtual worldTree-like structure for organising a virtual world

e.g. VRML

Hi h f d th t d fiHierarchy of nodes that define:Groups (and Switches, Sequences etc…)TransformationsProjectionsGeometry…

And states and attributes that define:Materials and texturesLighting and blending…

Scene Graph Example

Benefits of a Scene GraphP fPerformance

Structuring data facilitates optimizationoptimization

- Culling, state management, etc…

AbstractionAbstractionUnderlying graphics pipeline is hiddenLow-level programming (“how do I display this?”) replaced with high-l l t (“ h t d I t t

Image: sgi

level concepts (“what do I want to display?”)

About Open Scene Graphhttp://www.openscenegraph.org/Open-source scene graph implementationBased on OpenGLObject-oriented C++ following design pattern principlesUsed for simulation, games, research, and industrial projectsActive development community

Maintained by Robert Osfield~2000 mailing list subscribersDocumentation project: www osgbooks comDocumentation project: www.osgbooks.com

Uses the OSG Public License (similar to LGPL)

About Open Scene Graph (2)

Pirates of the XXI Century Flightgear SCANeR

3DVRII Research Institute EOR VRlab Umeå University3DVRII Research Institute EOR VRlab Umeå University

Open Scene Graph FeaturesPlugins for loading and saving

3D: 3D Studio (.3ds), OpenFlight (.flt), Wavefront (.obj)…2D: .png, .jpg, .bmp, QuickTime movies

NodeKits to extend functionalityye.g. osgShadow

Cross-platform support for:Cross platform support for:Window management (osgViewer)Threading (OpenThreads)Threading (OpenThreads)

Open Scene Graph Architecture

Inter-operability with other environments,

e.g. Python

Plugins read and NodeKits extend Plugins read and write 2D image and 3D model files

core functionality, exposing higher-level node types

Scene graph and rendering

functionalityfunctionality

Some Open Scene Graph Demos

osgviewer osgmotionblur osgparticle

You may want to get the OSG data package:Via SVN: http://www.openscenegraph.org/svn/osg/OpenSceneGraph-Data/trunk

osgreflect osgdistortion osgfxbrowser

Learning OSGCheck out the Quick Start Guide

Free PDF download at http://osgbooks.com/, Physical copy $13US

J i h ili li Join the mailing list: http://www.openscenegraph.org/projects/osg/wiki/MailingLists

Browse the website: http://www.openscenegraph.org/projects/osg

Use the forum: http://forum.openscenegraph.orgp p g p g

Study the examplesRead the source?

ARTosgART

What is osgART?

osgART adds AR to Open Scene GraphFurther developed and enhanced by:Further developed and enhanced by:

Julian LooserH S hHartmut SeichterRaphael Grasset

Current version 2.0, Open Sourcehttp://www osgart orghttp://www.osgart.org

osgART Approach: Basic Scene Graph

Root

T f0.988 -0.031 -0.145 0 -0.048 0.857 -0.512 0 [ ]Transform 0.141 0.513 0.846 0 10.939 29.859 -226.733 1 [ ]

3D ObjectTo add Video see-through AR:

Integrate live videogApply correct projection matrixUpdate tracked transformations in realtime

osgART Approach: AR Scene Graph

Root

Transform

3D Object

osgART Approach: AR Scene GraphRoot

Transform

VirtualCamera

VideoLayer

Video3D Obj

Layer

Geode 3D Object

osgART Approach: AR Scene GraphRoot

Projection matrix fromtracker calibration

Orthographic

Transform

VirtualCamera

Transformation matrix updated from marker tracking in realtimeVideo

Layer

Orthographic projection

Video3D Obj

LayerFull-screen quad with live texture updated from Video source

Geode 3D ObjectVideo source

osgART Approach: AR Scene GraphRoot

Projection matrix fromtracker calibration

Orthographic

Transform

VirtualCamera

Transformation matrix updated from marker tracking in realtimeVideo

Layer

Orthographic projection

Video3D Obj

LayerFull-screen quad with live texture updated from Video source

Geode 3D ObjectVideo source

osgART ArchitectureLike any video see-through AR library, osgART requires video input and tracking capabilities.

AVideo Sourcee g DirectShow A

R Lib

Applica

e.g. DirectShow

rary

ationTracking Module(libAR.lib)

osgART ArchitectureosgART uses a plugin architecture so that video sources and tracking technologies can be plugged in as necessary

OpenCVVideo

V

OpenCVVideo -VidCapture -CMU1394 -

PointGrey SDK -VidereDesign -

V d W

os

App

Video Plugin

VideoWrapper -VideoInput -

VideoSource -DSVL -

Intranel RTSP - sgART

plication

TrackeARToolKit4 -

ARToolkitPlus - ner Plugin

MXRToolKit -ARLib -

bazAR (work in progress) -ARTag (work in progress) -

Basic osgART TutorialD l ki ART li ti f t hDevelop a working osgART application from scratch.Use ARToolKit 2.72 library for tracking and video capturecapture

osgART Tutorial 1: Basic OSG ViewerStart with the standard Open Scene Graph ViewerWe will modify this to do AR!y

osgART Tutorial 1: Basic OSG Viewer

The basic osgViewer…#include <osgViewer/Viewer>#include <osgViewer/ViewerEventHandlers>#include <osgViewer/ViewerEventHandlers>

int main(int argc, char* argv[]) {

// Create a viewerosgViewer::Viewer viewer;

// Create a root nodeosg::ref_ptr<osg::Group> root = new osg::Group;

// Attach root node to the viewerviewer.setSceneData(root.get());g

// Add relevant event handlers to the viewerviewer.addEventHandler(new osgViewer::StatsHandler);viewer.addEventHandler(new osgViewer::WindowSizeHandler);viewer.addEventHandler(new osgViewer::ThreadingHandler);viewer addEventHandler(new osgViewer::HelpHandler);viewer.addEventHandler(new osgViewer::HelpHandler);

// Run the viewer and exit the program when the viewer is closedreturn viewer.run();

}

osgART Tutorial 2: Adding VideoAdd a video plugin

Load, configure, start video capture…

Add a video backgroundCreate, link to video, add to scene-graphg p

osgART Tutorial 2: Adding VideoNew code to load and configure a Video Plugin:

// Preload the video and tracker// Preload the video and trackerint _video_id = osgART::PluginManager::getInstance()->load("osgart_video_artoolkit2");

// Load a video plugin.osg::ref_ptr<osgART::Video> video =

dynamic_cast<osgART::Video*>(osgART::PluginManager::getInstance()->get(_video_id));

// Check if an instance of the video stream could be createdif (!video.valid()) {

// Without video an AR application can not work. Quit if none found.osg::notify(osg::FATAL) << "Could not initialize video plugin!" << std::endl;exit(-1);

}

// Open the video. This will not yet start the video stream but will// get information about the format of the video which is essential// for the connected tracker.video->open();video->open();

osgART Tutorial 2: Adding VideoNew code to add a live video background

osg::Group* createImageBackground(osg::Image* video) {osgART::VideoLayer* _layer = new osgART::VideoLayer();_layer->setSize(*video);osgART::VideoGeode* _geode = new osgART::VideoGeode(osgART::VideoGeode::USE_TEXTURE_2D, video);addTexturedQuad(*_geode, video->s(), video->t());_layer->addChild(_geode);return _layer;

}

osg::ref_ptr<osg::Group> videoBackground = createImageBackground(video.get());

In the main function…

videoBackground->getOrCreateStateSet()->setRenderBinDetails(0, "RenderBin");

root->addChild(videoBackground.get());video->start();

osgART Tutorial 3: TrackingAdd T k l Add a Tracker plugin

Load, configure, link to videoAdd a Marker to track

Load activateLoad, activateTracked node

C li k i h k i ki llb kCreate, link with marker via tracking callbacksPrint out the tracking data

osgART Tutorial 3: TrackingLoad a tracking plugin and associate it with the video plugin

int _tracker_id = osgART::PluginManager::getInstance()->load("osgart_tracker_artoolkit2");

osg::ref_ptr<osgART::Tracker> tracker = dynamic_cast<osgART::Tracker*>(osgART::PluginManager::getInstance()->get(_tracker_id));

oa a t ac g p ug a assoc ate t w t t e v eo p ug

if (!tracker.valid()) {// Without tracker an AR application can not work. Quit if none found.osg::notify(osg::FATAL) << "Could not initialize tracker plugin!" << std::endl;exit(-1);

}

// get the tracker calibration objectosg::ref_ptr<osgART::Calibration> calibration = tracker->getOrCreateCalibration();

// load a calibration fileif (!calibration->load("data/camera para dat"))if (!calibration->load( data/camera_para.dat )) {

// the calibration file was non-existing or couldnt be loadedosg::notify(osg::FATAL) << "Non existing or incompatible calibration file" << std::endl;exit(-1);

}

// set the image source for the trackertracker->setImage(video.get());

osgART::TrackerCallback::addOrSet(root.get(), tracker.get());

// create the virtual camera and add it to the sceneosg::ref_ptr<osg::Camera> cam = calibration->createCamera();root->addChild(cam.get());

osgART Tutorial 3: TrackingLoad a marker and activate itLoad a marker and activate itAssociate it with a transformation node (via event callbacks)Add the transformation node to the virtual camera node

osg::ref_ptr<osgART::Marker> marker = tracker->addMarker("single;data/patt.hiro;80;0;0");if (!marker.valid()) {

// Without marker an AR application can not work. Quit if none found.osg::notify(osg::FATAL) << "Could not add marker!" << std::endl;osg::notify(osg::FATAL) << "Could not add marker!" << std::endl;exit(-1);

}

marker->setActive(true);

osg::ref_ptr<osg::MatrixTransform> arTransform = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransform.get(), marker.get());cam->addChild(arTransform.get());

Add a debug callback to print out information about the tracked marker

osgART::addEventCallback(arTransform.get(), new osgART::MarkerDebugCallback(marker.get()));

Add a debug callback to print out information about the tracked marker

osgART Tutorial 3: Tracking

Tracking information is goutput to console

osgART Tutorial 4: Adding ContentN t the trackin data t se!Now put the tracking data to use!Add content to the tracked transformBasic cube code

arTransform->addChild(osgART::testCube());arTransform->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");

osgART Tutorial 5: Adding 3D ModelO S G h l d 3D f di lOpen Scene Graph can load some 3D formats directly:

e.g. Wavefront (.obj), OpenFlight (.flt), 3D Studio (.3ds), COLLADAOthers need to be convertedOthers need to be converted

Support for some formats is much better than otherse.g. OpenFlight good, 3ds hit and miss.

Recommend native .osg and .ive formats.osg – ASCII representation of scene graphive Binary OSG file Can contain hold textures.ive – Binary OSG file. Can contain hold textures.

osgExp : Exporter for 3DS Max is a good choicehttp://sourceforge.net/projects/osgmaxexp

Otherwise .3ds files from TurboSquid can work

osgART Tutorial 5: Adding 3D Model Replace the simple cube with a 3D model

std::string filename = "media/hollow_cube.osg";arTransform->addChild(osgDB::readNodeFile(filename));

Replace the simple cube with a 3D modelModels are loaded using the osgDB::readNodeFile() function

arTransform >addChild(osgDB::readNodeFile(filename));

3D St di M

Export to .osg

Note: Scale is important Units are in mm

3D Studio Max

osgARTNote: Scale is important. Units are in mm.

osgART Tutorial 6: Multiple Markers

Repeat the process so far to track more than one marker simultaneouslyy

osgART Tutorial 6: Multiple MarkersR h f k h k

Load and activate two markers

Repeat the process so far to track more than one marker

Load and activate two markersosg::ref_ptr<osgART::Marker> markerA = tracker->addMarker("single;data/patt.hiro;80;0;0");markerA->setActive(true);

osg::ref_ptr<osgART::Marker> markerB = tracker->addMarker("single;data/patt.kanji;80;0;0");_markerB->setActive(true);

Create two transformations, attach callbacks, and add models

osg::ref_ptr<osg::MatrixTransform> arTransformA = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformA.get(), markerA.get());arTransformA->addChild(osgDB::readNodeFile("media/hitl_logo.osg"));arTransformA->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformA.get());

osg::ref_ptr<osg::MatrixTransform> arTransformB = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformB.get(), markerB.get());arTransformB->addChild(osgDB::readNodeFile("media/gist_logo.osg"));arTransformB->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformB.get());

osgART Tutorial 6: Multiple Markers

Basic osgART Tutorial: Summary

Standard OSG Viewer Addition of Video Addition of Tracking

Addition of basic 3D graphics

Addition of 3D Model Multiple Markersg p

FLARMFLARManagerhttp://transmote com/flarhttp://transmote.com/flar

FLARManager:

Makes building FLARToolkit apps easierIs open-source, with a free and commercial licenseIs open source, with a free and commercial licenseIs designed to allow exploration of both augmented reality and alternative controllersreality and alternative controllersWas initiated by Eric Socolofsky, developed with

ib i f FLART lki icontributions from FLARToolkit communityDecouples FLARToolkit from Papervision3DConfiguration without recompilation, via xml config

FLARManager: features

Gives more control over application environmentProvides multiple input optionsRobust multiple marker managementRobust multiple marker managementSupports multiple 3D frameworksOff f f i i iOffers features for optimizationAllows for customization

Resources

Websites

Software Downloadhttp://artoolkit sourceforge net/http://artoolkit.sourceforge.net/

ARToolKit Documentationhttp://www.hitl.washington.edu/artoolkit/

ARToolKit ForumARToolKit Forumhttp://www.hitlabnz.org/wiki/Forum

ARToolworks Inchttp://www.artoolworks.com/p

ARToolKit PlusARToolKit Plushttp://studierstube.icg.tu-

t/h dh ld / t lkit l hgraz.ac.at/handheld_ar/artoolkitplus.php

osgARThttp://www.osgart.org/

FLARToolKitFLARToolKithttp://www.libspark.org/wiki/saqoosha/FLARToolKit/

FLARManagerhttp://words transmote com/wp/flarmanager/http://words.transmote.com/wp/flarmanager/

BooksI E h O S Interactive Environments with Open-Source Software: 3D Walkthroughs and Augmented R li f A hi i h Bl d 2 43 DART Reality for Architects with Blender 2.43, DART 3.0 and ARToolKit 2.72 by Wolfgang Höhl

A Hitchhikers Guide to Virtual Reality by Karen y yMcMenemy and Stuart Ferguson

More Information

M k Billi h t• Mark Billinghurst– mark.billinghurst@hitlabnz.org

• Websiteshi l b– www.hitlabnz.org