Author(ing)

Author, auctorem : "enlarger, founder" or literally “someone causing growth"

Programmer Designer

Content Driven

Tool Driven

Authoring in 3D

Wireframe

3-Window

Keyboard

Pen

…

New Possibilities with AR

Authoring & Content Design– Problem Framing

• Analyze the reported “problem”– Task Analysis

• Domain: common themes, interactions • Objectives: goal of the user• Limitations: identify limits

– Structuring/Segmentation• Real World• Virtual World

– Description and Meaning• Logical Connection between elements

Problem Framing: Wicked Problem

• Introduced by Horst Rittel and Melvin Webber:– No definition– Ill-defined problems– Pre-empted solutions– Circular problem dependencies (no stopping

condition)

“The Problem”“The Problem”

Problem Framing

Identify all angles of a problem

Restate the Problem

Reformulate

Identify Sub-problems

Problem Solving

• Discovering Design, Richard Buchanan, Victor Margolin, University of Chicago Press

SetupSetup

InteractionInteraction

ErgonomicsErgonomics

ContextualCoherenceContextualCoherence

Physical and Virtual

Content Design

Physical and Virtual

Content Design

Physical

Virtual

Create an interactive, meaningful experience

without the need of programming!

Small recap

• Traditional 3D content creation can not foster the possibilities of AR

• Developing AR application needs extra attention on solving the right problem

• Merging virtual and real environments needs attention on the domain, context and the goals of the user

Aspects of Authoring• Domain Awareness

– Entertainment– Design– Art – Engineering– Education…

Augmented Reality IssuesRegistration

Relation between real and virtual world

Occlusionawareness of parts of the scene that are not augmented

Impact of Augmentationchange meaning, obstruct or alter

Reality-Virtuality Relationship

• Contextrelation of virtual content and real world

• Perceptionvisual or aural appearance

• Ontologymeaning of augmentation

• Collaboration / Participation social impact and engagement

Aspect: Context

• Functional and visual context relationship

• Embedding of virtual content as extension or alteration

Picture Courtesy: Patrick Baudisch

Aspect: Context• Registration

– Degree of Freedom (DOF) used

• Interaction– Proximity– Visibility– Orientation– Occlusion

Functional Context in AR

Augmented Kitchen: Spatial Reality AR Soccer: Mobile AR

Context Spatial, Aiding Augmentation Context: mobile fiducial and

visualisation

Perceptual Context in AR

Using the contextual information to blend virtual content

Aspect: Perception

• Visualisation– Photorealistic– NPR

• Occlusion• Mapping of real world

events

Conceptual Realization

Aspect: Ontology

• Ontology: Meaning and nature of things– Affordance

• Relation of objects and subjects• James J. Gibbson (other variants by Husserl, Norman)

– Augmentation• Hide or enhance meaning• New meaning through overlay

Relevance of Affordance

WIMPWindow, Icons, Menu, Pointer

Collaboration an afterthought

Virtually no difference between text processor and 3D modeling tool

Affordance

• Property describing the relation of objects

• Gibson: Affordances (1979, General)

• Norman (1988): – Actual Affordances– Perceived Affordances

lets me throughcan be opened

perceived actual

Affordance

Input

Feedback

Aspect: Collaboration

• Affords– Sharing– Peer review

• User dependent views

Aspect: Collaboration

• Engagement– Involving– Modes (direct access)

• Interactivity• Sharing

– Policies– Locking

Aspect: Education

• Low cost infrastructure• Informal learning• Engagement• Peer learning

Tangible AR Design Principles

• Tangible AR Interfaces use TUI principles– Physical controllers for moving virtual content– Support for spatial 3D interaction techniques– Time and space multiplexed interaction– Support for multi-handed interaction– Match object affordances to task requirements– Support parallel activity with multiple objects– Allow collaboration between multiple users

Mixed Reality Continuum

Hypothesis– The next generation of interfaces will support transitions

along the Reality-Virtuality continuum

Reality(Tangible Interfaces)

Virtuality(Virtual Reality)

Augmented Reality (AR)

Augmented Virtuality (AV)

Mixed Reality (MR)

Milgram (1994)

The MagicBook

Design Goals– Allows user to move smoothly between reality

and virtual reality– Support collaboration

MagicBook Metaphor

Features

Seamless transition between Reality and Virtuality– Reliance on real decreases as virtual increases

Supports egocentric and exocentric views– User can pick appropriate view

Computer becomes invisible– Consistent interface metaphors– Virtual content seems real

Supports collaboration

Technology

• Reality– No technology

• Augmented Reality– Camera – tracking– Switch – fly in

• Virtual Reality– Compass – tracking– Press pad – move– Switch – fly out

Summary

• Context awareness is paramount• AR can alter, obscure or hinder depending on

the integration, visually or functional• AR can change meaning of objects: be aware

of perceived affordances• AR is not necessarily standalone: transitional

methods can help to blur boundaries

Traditional 3D Content Design and AR

• No context awareness• No feedback within the

real environment• Limited interactivity of

content

Challenges for AR authoring

• Store contextual information• Define relationships between real and virtual

environments• Override management: virtual or real first

priority in interaction (e.g. physics)• Feedback loop: prototype interaction

metaphors

Authoring Approaches

• Build within– Develop concepts and

content on the fly– Merged runtime and

design time

• Design and Run– Develop concepts and

content externally– Divide runtime and

design time

Tools for AR Development

• ToolkitsComputer VisionRendering

• FrameworksInfrastructureModel Loader

• Authoring ToolsGUIInteraction Design

High Level

Low Level

Low Level Toolkits

AR Toolkithttp://artoolkit.sourceforge.net/

MXR Toolkithttp://mxrtoolkit.sourceforge.net/

MR Toolkit

Frameworks

Studierstubehttp://studierstube.icg.tu-graz.ac.at

DWARFhttp://ar.in.tum.de/Chair/ProjectDwarf

VHD++http://vhdpp.sourceforge.net/

Example: Studierstube

AR Authoring

AMIREhttp://www.amire.net/index.html

DARThttp://www.gvu.gatech.edu/dart/

APRILhttp://studierstube.icg.tu-graz.ac.at/april/

Audiences for Authoring Tools

• Content creation and assembly for a wide audience

• Unspecific about the supported interactions

• Specific for a target audience

• Support domain related context

Generic Authoring

Designer AR Toolkit: DART

Using Director as a base for AR authoring and utilizing the infrastructure of the tool it embeds in

AMIREGeneric authoring tool to create AR content through a GUI

Domain Specific

AR formal design: sketchand+

virtual inside real

creation in virtual

sharing local/remote

Archaeological AR (Columbia University)Planning and developing digging strategies

INSCAPE AR (www.inscapers.com)Authoring of AR content for interactive story telling

Authoring and Storage

• Traditional– VRML (original MagicBook)– OpenFlight– 3D studio– Maya / Studio Tools

• AR related– APRIL

VRML (Virtual Reality Markup Language)• Originates sgi Open Inventor format

– Replication of Scenegraph structure• Version 1.0• Version 2.0 (VRML97) ISO standard

– Support for videotextures, sensors etc.– New syntax

• X3D– XMLyfied VRML97– Programmable shaders– Extensible through XML namespaces

• Check http://web3d.org

VRML Advantages

• Geometry• Textures• Animation• Hyperlinking

Disadvantages

• No support yet for context awareness– Web3D consortium is discussing an extension to

X3D to change this (First proposal submitted in 2007)

• Only very limited support in non-commercial libraries

• Hint on the side: Don’t use the VRML support in AR Toolkit

Summary

• AR content can be developed: from within and externally

• Like in traditional 3D content development there is a differentiation between domain specific and generic tools

• Existing 3D storage solutions do not honour specifics of AR

ComposARPython based tool for AR authoring. Supporting various interaction techniques and input

ComposAR

• Persistant storage for Scenes• Cross platform consistent GUI• Import from various 3D content creation tools

(Maya, 3ds max etc)• Highly configurable

Our Approach

System

osgART

osgSWIG

Utility

ComposAR

OpenSceneGraph

osgSWIG

• Thin Meta API for OpenSceneGraph and osgART

• Wrapper for– Java, C#, Python, Ruby, Lua

• WrapperWrapper for osgIntrospection– Transparent cross-language Script within Script

Fin

Questionshartmut.seichter@hitlabnz.org