Sketch­based­interface­on­a­handheld­augmented­reality­system­

Rhys­MoyneHonours­Minor­Thesis

Supervisor:­Dr.­Christian­Sandor

TINT

Augmented­Reality­(AR)

Adding­virtual­information­to­the­real­world Aids­user­in­understanding­the­world Merges­real­world­(camera­image)­and­virtual­

objects Tracking­is­needed­to­calculate­location­of­

objects­and­user­in­the­environment

AR­Display­Technologies

Head-mounted Handheld

Motivation

Handheld­AR­systems­require­different­ways­of­interaction­due­to­size­constraint

Exisiting­methods­such­as­pinch­gloves­or­wrist­pads­are­not­suitable

Future­AR­devices­such­as­mobile­phones­are­likely­to­have­touch­screens

Sketch-based­input­in­handheld­augmented­reality­is­largely­unexplored

Direct­Manipulation­and­Sketch-based­Interface

Direct­Manipulation

Direct­Manipulation

User­actions­affect­object­immediately Example:­driving­a­car

User­turns­steering­wheel­left­and­wheel­moves­left No­command­“TURN­LEFT”

“Matching­user's­gestures­with­the­observed­virtual­motion”­(Dragicevic­et­al.,­p.­2)

Sketch-based­interface

Sketch-based­interface

Allows­user­to­directly­interact­similar­to­pen­and­paper More­natural Has­been­explored­in­areas­such­as­modelling,­

animation,­user­interface­prototyping Gestures

Research­Question

Is­a­sketch­based­interface­a­suitable­interaction­method­in­a­handheld­augmented­reality­system?

Research­Approach

Literature­Review Iterative­prototypes

Informal­qualitative­feedback­on­techniques (User­study)

Development­Approach

Create­a­demo­illustrating­interactive­exploration­in­a­simple­test­scene Make­use­of­TINT­framework

Add­tracking Add­animation

Direct­manipulation­interface­(complete­by­18­September) More­complicated­physics­(18­September) Gesture­Recognition­(18­September)

Port­to­Mobile­Augmented­Reality­(30­September) Projection­of­camera­image­onto­virtual­objects

Demo

This­can­be­used­as­a­base­to­control­visualisations­in­TINT

Illustrates­how­to­control­objects­to­explore

TINT­(This­is­not­TINMITH)

TINMITH­–­HMD­prototyping­platform TINT­–­Handheld­augmented­reality­prototyping­

platform Used­to­prototype­AR­applications­that­may­be­

possible­on­future­mobile­phones Written­in­Python­allowing­fast­development

Implementation

ARToolkit­Tracking

Calculates­where­the­marker­is­relative­to­the­camera­position

Uses­computer­vision­techniques­to­identify­marker

Allows­the­virtual­objects­to­be­placed­in­correct­location

Compiz­Physics

$1­Gesture­Recognizer

Summary

How­to­interactively­control­visualisations­in­handheld­augmented­reality?

Thanks,­Questions?

ReferencesCompiz.org,­2009,­'Compiz',­<http://www.compiz.org/>,­accessed­3­September­2009.

Dragicevic,­P,­Ramos,­G,­Bibliowitcz,­J,­Nowrouzezahrai,­D,­Balakrishnan,­R­&­Singh,­K­2008,­'Video­

­­­­­­­browsing­by­direct­manipulation',­in­Twenty­sixth­annual­SIGCHI­conference­on­Human­factors­in­­

­­­­­­­computing­systems,­ACM,­Florence,­Italy,­pp.­237­246.

Igarashi,­T,­Matsuoka,­S­&­Tanaka,­H­1999,­'Teddy:­a­sketching­interface­for­3D­freeform­design'­in­

­­­­­­­­Proceedings­of­the­26th­annual­conference­on­computer­graphics­and­interactive­techniques,­ACM,­

­­­­­­­­pp.­409­416.

Kato,­H­&­Billinghurst,­M­1999,­'Marker­tracking­and­HMD­calibration­for­a­video­based­augmented­reality­

­­­­­­­­conferencing­system',­in­Proceedings­of­Augmented­Reality­1999­(IWAR­'99),­IEEE,­San­Francisco,­

­­­­­­­­CA,­pp.­85­94.

Piekarski,­W­&­Thomas,­BH­2001,­'Tinmith­Metro:­new­outdoor­techniques­for­creating­city­models­with­an­

­­­­­­­­augmented­reality­wearable­computer',­in­Proceedings­of­Fifth­International­Symposium­on­­

­­­­­­­­Wearable­Computers,­IEEE,­Zurich,­pp.­31­38.

Sandor,­C,­Cunningham,­A,­Eck,­U,­Urquhart,­D,­Jarvis,­D,­Dey,­A,­Barbier,­S,­Marner,­M­&­Rhee,­S­2009,­

­­­­­­­'Egocentric­space­distorting­visualizations­for­rapid­environment­exploration­in­mobile­mixed­reality',­

­­­­­­­in­8th­IEEE­and­ACM­­International­Symposium­on­Mixed­and­Augmented­Reality­(ISMAR'09),­ACM,­

­­­­­­­Orlando,­Florida.

Shneiderman,­B­1983,­'Direct­Manipulation:­a­step­beyond­programming­languages',­Computer, vol.­16,­no.­8,­pp.­57-69.

sleepygeek.org,n.d.,­­'$1­gesture­recognizer­in­python',­<http://sleepygeek.org/projects.dollar>,­accessed­3­September­2009.

Wobbrock,­JO,­Wilson,­AD­&­Li,­Y­2007,­'Gestures­without­libraries,­toolkits­or­training:­a­$1­recognizer­for­

­­­­­­user­interface­prototypes'­in­Proceedings­of­the­20th­annual­ACM­symposium­on­user­interface­­

­­­­­­software­and­technology,­ACM,­Newport,­Rhode­Island,­pp.­159­168.