Hybrid User Interfaces for Augmented Reality
-
Upload
disruptive-code -
Category
Technology
-
view
2.381 -
download
2
description
Transcript of Hybrid User Interfaces for Augmented Reality
![Page 1: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/1.jpg)
Hybrid User Interfaces for Augmented Reality
Alex Olwal, Ph.D. olwal.com
Human-Computer Interaction, KTH
![Page 2: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/2.jpg)
Alex Olwal, Ph.D.
2003-2010 KTH Stockholm
2006 Microsoft Research Redmond
2005 University of California Santa Barbara
2002-2003 Columbia University New York
startup Rule Communication Stockholm www.rule.se
startup Harmonic Solutions Stockholm www.harmonic.se
Display Systems
Sensing
Interaction Techniques
![Page 3: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/3.jpg)
WIKITUDE
Layar
Augmented Reality (AR) on mobile phones
![Page 4: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/4.jpg)
X: The Man with the X-Ray Eyes (1963)
![Page 5: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/5.jpg)
RoboCop (1987)
![Page 6: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/6.jpg)
Terminator 3 (2003)
![Page 7: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/7.jpg)
Augmented Reality (AR)
• Empower our senses → enhance perception
• Supplement real
• Seamless
![Page 8: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/8.jpg)
AR Fuse virtual objects with real environment
• View
• Perspective
• Real-time
![Page 9: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/9.jpg)
Head-worn
Ivan Sutherland 1968 5dt HMD Nomad ProView XL40/50 STm AddVisor 150
![Page 10: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/10.jpg)
Head-worn
Handheld, Situated & Hybrid
Everywhere Displays [Pinhanez et al. 2001], iLamp Projectors [Raskar et al. 2003],On-board Mixed Reality Projector [Karitsuka & Sato 2003], Transflective Surfaces [Bimber et al. 2000], Cell phone [Möhring & Bimber 2004], NaviCam [Rekimoto 1997], PDA [Wagner & Schmalstieg 2003]
![Page 11: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/11.jpg)
Ubiquitous computingWeiser 1993
Ubiquitous
• Displays
• Sensing
• Connectivity
Devices
• Tabs, pads & boards
![Page 12: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/12.jpg)
Interactive surfaces & tangible UIs
• Display 2D graphics
• Sense on & above surface
• Detect & augment objects
DigitalDesk [ Wellner 1993 ]
Tangible Bits [ Ullmer & Ishii & Buxton 1997 ]
Augmented Surfaces [ Rekimoto & Saitoh 1999 ]
PlayAnywhere / PlayTogether [ Wilson 2005 / Wilson & Robbins 2006 ]
![Page 13: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/13.jpg)
• Tracked display
• Context-sensitive
• Focus + context
Chameleon [ Fitzmaurice 1993 ]NaviCam [ Rekimoto 1995 ]Augmented Notebook [ Mackay et al. 2002 ]Focus + Context displays [ Baudisch et al. 2002 ]VITA [ Benko et al. 2004 ]Ubiquitous graphics [ Sanneblad & Holmquist 2006 ]
Spatially aware displays
![Page 14: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/14.jpg)
14
NaviCamRekimoto 1995
![Page 15: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/15.jpg)
Hybrid UIs for AR
1 Unobtrusive
2 Sensing in environment
AR, UbiComp, Mobile, Interactive Surfaces, …
Shopping window Microsoft Surface
![Page 16: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/16.jpg)
SurfaceFusionSensing & augmenting everyday objects
GI 2008 [Olwal & Wilson]Proc. Graphics Interface
![Page 17: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/17.jpg)
LightSenseDynamic augmentations of printed media
ISMAR 2006 [Olwal]Proc. IEEE & ACM International Symposium on Mixed & Augmented Reality
Ericsson Demo CenterStockholm 2008
![Page 18: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/18.jpg)
LUMARHybrid 2D + 3D handheld AR
ICAT 2007 [Olwal & Henrysson]Proc. International Conference on Artifical Reality & Teleexistence
Judge’s Choice Award in Nokia “Calling All Innovators” Challenge 2009Mobile World Congress, Barcelona, 13-16 February, 2009
![Page 19: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/19.jpg)
LUMARHybrid 2D + 3D handheld AR
Printed media
Floor plan
2D graphics
Photographs
3D graphics
3D model
![Page 20: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/20.jpg)
Spatially aware handheldsEnhancing interaction with large displays
TEI 2009 [Olwal & Feiner]Proc. International Conference on Tangible & Embedded Interaction
INTERACT 2009 [Olwal]Proc. IFIP TC13 Conference on Human-Computer Interaction
Ericsson Trade Show Events in 2009 / 2010Barcelona, Las Vegas, Boston, Galway, Stockholm, Paris, Amsterdam, San Francisco
![Page 21: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/21.jpg)
352 px
288
px
408 mm
306
mm
800 px
600
px
71 px
95 p
x
camera view
projection & touch input
mobile device 2704 mobile px
2021
mob
ile p
xdevice tracking region
improved visual fidelity & touch input resolution
projected pixels
mobile pixels
240 px
36 mm
48 m
m
320
px
217 px
163
px
![Page 22: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/22.jpg)
px/mm2
0
20
40
60
80
100
120
140
160
180
Projector32"
1024×768
LCD21"
1600×1200
Apple iPhone3.5"
320×480
Sony Ericsson G9002.4"
240×320
Nokia 5800XM3.2"
360×640
Google Nexus One3.7"
480×800
Apple iPhone 43.5"
640x960
ResolutionDenser pixels & sensors on handhelds
![Page 23: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/23.jpg)
Augmenting surface interaction Mobile devices ・ expanded capabilities
INTERACT 2009 [Olwal]Proc. IFIP TC13 Conference on Human-Computer Interaction
Ericsson Trade Show Events in 2009 / 2010Barcelona, Las Vegas, Boston, Galway, Stockholm, Paris, Amsterdam, San Francisco
![Page 24: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/24.jpg)
• Need HybridSurface slide
Augmenting surface interactionCollaboration for multiple users, devices & locations
TEI 2009 [Olwal & Feiner]Proc. International Conference on Tangible & Embedded Interaction
INTERACT 2009 [Olwal]Proc. IFIP TC13 Conference on Human-Computer Interaction
Ericsson Trade Show Events in 2009 / 2010Barcelona, Las Vegas, Boston, Galway, Stockholm, Paris, Amsterdam, San Francisco
![Page 25: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/25.jpg)
Hybrid UIs for AR
1 Unobtrusive
2 Sensing in environment
3 Novel display technology
AR, UbiComp, Mobile, Interactive Surfaces, …
Shopping window Microsoft Surface Microvision HUD Heliodisplay, Mid-air
![Page 26: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/26.jpg)
ASTORTransparent window with autostereoscopic 3D overlays
SPIE 2008 [Olwal, Gustafsson & Lindfors]Proc. SPIE Electronic Imaging, Vol 6804
ISMAR 2005 [Olwal, Lindfors, Gustafsson, Kjellberg & Mattson]Proc. IEEE & ACM Symposium on Mixed & Augmented Reality
SIGGRAPH 2004 Sketches [Olwal, Lindfors & Gustafsson]International Conference on Computer Graphics & Interactive Techniques
![Page 27: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/27.jpg)
![Page 28: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/28.jpg)
workspace
process data
![Page 29: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/29.jpg)
workspace & process data
![Page 30: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/30.jpg)
Unencumbered 3D interactionSupporting walk-up-and-use for public displays
NordiCHI 2008 [Olwal]Proc. Nordic Conference on Human Computer Interaction
![Page 31: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/31.jpg)
eye tracker
touch overlay
transparent 3D display ( holographic optical element )
3D camera
![Page 32: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/32.jpg)
Unencumbered 3D interactionManipulate 3D data using mobile, gestures & touch
NordiCHI 2008 [Olwal]Proc. Nordic Conference on Human Computer Interaction
![Page 33: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/33.jpg)
3DTV Book 2008 [DiVerdi, Olwal, Rakkolainen & Höllerer]Three-Dimensional Television: Capture, Transmission & Display, Springer
VR 2006 [Olwal, DiVerdi, Rakkolainen & Höllerer]Proc. IEEE Virtual Reality
SIGGRAPH 2005 Emerging Technologies [Rakkolainen, DiVerdi, Olwal, Candussi & Höllerer]International Conference on Computer Graphics & Interactive Techniques
Immaterial displaysFace-to-face, reach-through & walk-through interaction
![Page 34: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/34.jpg)
Hybrid UIs for AR
1 Unobtrusive
2 Sensing in environment
3 Novel display technology
4 Hybrid approaches → sensing / displays / interaction
AR, UbiComp, Mobile, Interactive Surfaces, …
BiDi: Flat depth-sensing displaysHirsch, Lanman, Holtzman & Raskar 2009
SecondLightIzadi, Hodges, Taylor, Rosenfeld, Villar, Butler & Westhues 2008
Interactions in the airHilliges, Izadi, Wilson, Hodges, Mendoza & Butz 2009
![Page 35: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/35.jpg)
Nintendo WiimoteCamera + accelerometers + gyros + “sensor” bar
![Page 36: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/36.jpg)
PlayStation MovePS Eye camera + Motion controller
![Page 37: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/37.jpg)
Microsoft KinectDepth camera
![Page 38: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/38.jpg)
Mobile projection
![Page 39: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/39.jpg)
AR marketing
![Page 40: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/40.jpg)
• Relevant
• Content consumption
• Interactivity
• Activity analysis
Rule Communication – www.rule.se
![Page 41: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/41.jpg)
HybridSurface – www.harmonic.se
• Hybrid interaction: Touch surface + mobile devices
• Ultra-thin design
• Mobile World Congress 2009 (Barcelona), CTIA Wireless 2009 (Las Vegas), Ericsson Volvo Ocean Race VIP Pavilion 2009 (Boston, Galway, Stockholm), Ericsson Experience Center
• Next generation for new Maritime Museum Exhibit 2011
![Page 42: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/42.jpg)
Acknowledgements
Organizers
• Disruptive Media
Collaborators
• Department of Production Engineering, KTH
• Center for Medicine, Health and Technology, KTH / KI / KS
• FunkIS (HCI + radiology research)
• Department of Mechatronics
• VITA, Linköping University
• Computer Graphics & UI lab, Columbia University
• ilab, USCB
Funding
• KK foundation
• Swedish Institute for Assistive Technologies
• Engineer’s Science Academy / Innovation Bridge
• Blanceflor Foundation
• Swedish Research Council
Equipment, donations & funding
• Ericsson, Nokia, Microsoft Research, Mitsubishi, Doro, …
![Page 43: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/43.jpg)
Hybrid User Interfaces for Augmented Reality
Alex Olwal, Ph.D.olwal.com
research: olwal.come-marketing: rule.se
new user interfaces : harmonic.se
![Page 44: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/44.jpg)
![Page 45: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/45.jpg)
![Page 46: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/46.jpg)
![Page 47: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/47.jpg)
![Page 48: Hybrid User Interfaces for Augmented Reality](https://reader034.fdocuments.us/reader034/viewer/2022052618/554a2616b4c9051b578b48c8/html5/thumbnails/48.jpg)
Hybrid User Interfaces for Augmented Reality
Alex Olwal, Ph.D.olwal.com
research: olwal.come-marketing: rule.se
new user interfaces: harmonic.se