GETTING REAL ABOUT AR Mark Billinghurst mark.billinghurst@unisa.edu.au

August 26th 2015

50 Years of Progress

• From lab to living room • AR devices in every pocket

1998 vs. 2008

CPU: 300 Mhz HDD; 9GB RAM: 512 mb Camera: VGA 30fps Graphics: 500K poly/sec

1998: SGI O2 2008: Nokia N95

CPU: 332 Mhz HDD; 8GB RAM: 128 mb Camera: VGA 30 fps Graphics: 2m poly/sec

AR Business Today

• Around $600 Million USD in 2014 (>$2B 2015) • > 80% Games and Marketing applications

Market Projections

cf. 2014 computer game market = $84 Billion

Crossing the Chasm

http://www.gartner.com/newsroom/id/3114217

Getting from Here to There • New markets

• Medical, Education, Industry • New types of applications

• Training, Information Presentation, Collaboration • New devices/platforms

• Head mounted displays, projected AR • New Interaction Techniques

• Gesture based, empathic interfaces, multimodal

2012 – Iron Man 2

AR for Industrial Training

• Potential Benefits •  Training on target equipment • Showing spatial information in place • Allow worker to remain focused on work site • Able to view content from any angle

Case Study 1: Vehicle Maintenance

• Using AR for armoured vehicle maintence

Henderson, S. J., & Feiner, S. (2009, October). Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In ISMAR 2009. 8th IEEE International Symposium on (pp. 135-144). IEEE.

Current System

• Electronic Manual on laptop computer • 2D images, pdf, fixed views

Interfaces

• Three display conditions • AR, HUD, LCD panel • Using stereo video-see through AR HMD system

AR HUD LCD

Experiment • 6 Mechanics as subjects • Subjects completed 18 common tasks

•  Installing fasteners, connecting cables, etc

• Measured • Performance time, localisation time • Errors made • Head movement, exertion • Subjective feedback (satisfaction, intuitiveness)

Key Results • No difference in times between AR & LCD

• But LCD (34.5s) faster than HUD (55.2s) • AR able to locate task areas faster

• 4.9s in AR, vs 9.2s in LCD, vs. 11.1s in HUD • Fewer head motions in AR

• Typically 50% less motions than other conditions • AR rated as good as LCD in satisfaction • AR rated as most intuitive of conditions

Case Study 2: Boeing & Iowa State

• Work instructions presented in 3 conditions •  Desktop interface •  Tablet interface •  Tablet AR interface

T. Richardson, S. Gilbert, J. Holub, F. Thompson, A. MacAllister, R. Radkowski, E. Winer, P. Davies, and S. Terry. "Fusing Self-Reported and Sensor Data from Mixed-Reality Training."

Desktop Interface Tablet AR Interface

Experiment Design • Reproduced industrial work cell

•  Task •  Subjects completed a five-step physical assembly process

• Data Collected •  User motion, activity analysis •  Task performance (accuracy, completion time) •  Time looking at instructions, moving between work areas •  Subjective feedback

Results

• Users liked AR condition the best •  Fewer errors in AR condition, faster performance time

•  AR has 30% faster performance, 90% few errors

Performance Time Errors

Commercial Systems • Ngrain

• http://www.ngrain.com/ • Training authoring tool • Model based AR tracking

• ScopeAR • http://www.scopear.com/ • Remote assistance •  Image based tracking

Ngrain

Intelligent Training • Most AR systems stupid

• Don’t recognize user behaviour • Don’t provide feedback • Don’t adapt to user

• Especially important for training • Scaffolded learning • Moving beyond check-lists of actions

Intelligent Interfaces

• AR interface + intelligent tutoring system • ASPIRE constraint based system (from UC) • Constraints

•  relevance cond., satisfaction cond., feedback

Westerfield, G., Mitrovic, A., & Billinghurst, M. (2013). Intelligent Augmented Reality Training for Assembly Tasks. In Artificial Intelligence in Education (pp. 542-551). Springer Berlin Heidelberg.

Domain Ontology

Intelligent Feedback

•  Actively monitors user behaviour •  Implicit vs. explicit interaction

•  Provides corrective feedback

Evaluation Results

• 16 subjects, with and without ITS • Improved task completion

• Improved learning

Intelligent Agents

• AR characters • Virtual embodiment of system • Multimodal input/output

• Examples • AR Lego, Welbo, etc • Mr Virtuoso

•  AR character more real, more fun • On-screen 3D and AR similar in usefulness

Wagner, D., Billinghurst, M., & Schmalstieg, D. (2006). How real should virtual characters be?. In Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology (p. 57). ACM.

Conclusions • AR is becoming commonly available

•  In order to achieve significant growth AR needs to • Expand into new markets • Move onto new platforms • Create new types of applications

• AR for Training is a particularly promising area • Spatial skills, intelligent interfaces

www.empathiccomputing.org

@marknb00

mark.billinghurst@unisa.edu.au