Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial...
Transcript of Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial...
![Page 1: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/1.jpg)
Mensch-Maschine-Interaktion 2
Interactive Environments
Prof. Dr. Andreas Butz, Dr. Julie Wagner
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 1
![Page 2: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/2.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 2
context and task
challenges
input technologies
challenges in interaction design
output technologies
Interactive Environments
![Page 3: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/3.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Display: Front Projection• what we are doing here in class• simplest way to produce visual
output on any surface• pro:
– cheap, simple– even light distribution– no additional space needed– space for legs under the table
• contra– interacting hand and person cast a shadow– only feasible for tabletops when firmly mounted
3
image source: http://www.rosco.com/
![Page 4: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/4.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Display: Rear Projection• Pro:
– projector is hidden, space in front empty– no shadowing of the surface
• Contra:– Can only be done with space behind– complex mirror construction for tabletops– can create „hot spot“ with cheap screen
4
image source: http://www.rosco.com/
![Page 5: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/5.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Display: Projection from the side ;-)• PlayAnywhere, Andy Wilson (Microsoft Research), 2005• Uses commercial short throw projector for front
projection at an angle of 40 degrees• Uses cameras for sensing
– mounted off axis from the projection– can see shadows caused by front projection– can recognize fingers and markers
• Turns any flat surface (e.g., table) into an interactive surface
5
![Page 6: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/6.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 6
![Page 7: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/7.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Display: Screens• What we initially used in our tabletop research @LMU
– High resolution and contrast + great color– Insensitive to ambient light– Can be bought with touch overlay for sensing
7
![Page 8: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/8.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 8
FogScreen
• A wall sized, immaterial projection display
• Image projected on dry, non-hazardous fog (pure water)
• Inventor: Ismo Rakkolainen
![Page 9: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/9.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 9
FogScreen (2)
• Fog sandwiched between airstreams
• Immaterial ! user can reach or walk through unhindered
![Page 10: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/10.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 10
FogScreen (3)
• Technical Details– First introduced at UIST
2004– So far only 9 FogScreens
in the world– Price: 100k $– Weight: around 100 kg– Needs: 10L/h, 300W
![Page 11: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/11.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 11
FogScreen (4)• Pseudo 3D
– Double sided projection possible– E.g. opposite views of the same scene
![Page 12: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/12.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 12
FogScreen(4)• Pseudo 3D – HeadTracking (S. DiVerdi)
– Tracking the users Head– Input used for adapting the frustum for accurate perspective
rendering
![Page 13: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/13.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 13
Everywhere Display Projector (IBM)http://www.research.ibm.com/ed/
Claudio Pinhanez
![Page 14: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/14.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 14
Everywhere display (cont.)
Output: a projector and a rotating mirror Input: a camera for interaction, NOT for image rectification!
![Page 15: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/15.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 15
Undistorting the projected image
" Place original image in the 3D model
" Virtual camera image shows it distorted
" Project the distorterd image from 3D model with the Real projector into the real world
– Distortions cancel each other out IF virtual camera and real projector are in the same location
![Page 16: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/16.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 16
context and task
challenges
input technologies
challenges in interaction design
output technologies
Interactive Environments
![Page 17: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/17.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Introduction and History• Early Research
– The MIT MetaDesk– Pierre Wellner‘s Digital Desk
17
![Page 18: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/18.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Historic Interactive Surfaces• read http://www.billbuxton.com/
multitouchOverview.html !• early experiments with multi touch in the 1980ies• For this lecture: 2 prominent historic examples:
– Pierre Wellner‘s Digital Desk– MIT MetaDesk
18
![Page 19: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/19.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Pierre Wellner‘s Digital Desk• Working prototype in 1991• Regular table with top projection• Overhead camera to detect fingers• Camera can also scan paper on the desk• Interaction with printed paper and digital
applications on the same surface
19
![Page 20: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/20.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 20
![Page 21: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/21.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
The MIT MetaDESK• Platform for exploring Tangible UIs (Ullmer & Ishii, 1997)• Also uses top projection• Various projects built on top of it
21
![Page 22: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/22.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Problems and Particularities• Asymmetric bimanuality• Territoriality on tables • Direction and orientation on tables
• Midas Touch Problem • Occlusion Problem• Fat Finger problem
22
![Page 23: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/23.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Territoriality on tables (Scott 2004)
• Studies on how people use the space on a table– puzzle, game, Lego activities + room planning on round tables
• Different areas on the table surface– personal space (directly in front of person)– group space (reachable by all members)– storage space (in the periphery)
• Boundaries between areas are flexible
23
![Page 24: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/24.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Territoriality on tables (Scott 2004)
24
![Page 25: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/25.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Territoriality on tables (Scott 2004)
• Design Implications:– Provide visibility and transparency of action– Provide appropriate table space– Provide functionality in the appropriate locality– Allow casual grouping of items and tools in the workspace
25
![Page 26: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/26.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Orientation on tables (Kruger 2003)
• Basic problem: no clearly defined „up“ direction when interacting with multiple users around a table
• Known approaches:– Fixed orientation– Manual orientation– Person-based automatic orientation– Environment-based automatic orientation
26
AB
CAB
C
![Page 27: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/27.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Orientation on tables (Kruger 2003)
• Orientation can serve as a collaborative resource: – Using someone else’s alignment conveyed support – Orientation could establish the intended audience – Orientation was also used to create a personal space.
27
![Page 28: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/28.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Orientation on tables (Kruger 2003)
• 3 main roles of orientation:• Comprehension
– Ease of reading – Ease of task – Alternate perspective
• Coordination – Establishment of personal spaces – Establishment of group spaces – Ownership of objects
• Communication – Intentional communication – Independence of orientation
28
![Page 29: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/29.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Midas Touch Problem• General problem with touch input:
– positioning and selection is the same– cannot position the pointer without selecting – there is no „hover“ state
• Name goes back to king Midas– everything he touched turned to gold– ...also food, so he was in danger to starve...
• Straightforward solution: dwell time– trigger a function only after a certain time– introduces artificial delays, disturbs the flow
• Also well-known in gaze-based interaction (eye tracking)29
http://de.wikipedia.org/wiki/Midas
![Page 30: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/30.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Dealing with the Midas Touch Problem• Lift-off techniques (e.g., Potter et al. CHI 1988)
– cursor is offset from the finger and displayed during exploration– function is only triggered when the finger leaves the surface– can be observed in the video on „shift“, earlier in this class
• Pressure sensing (e.g., Richter et al. AutoUI 2010)– finger can explore the screen in hover state– selection by pressing harder
30
![Page 31: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/31.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Interaction Techniques on Int. Surfaces• Motivation: thinking about bumptop• Single touch
– RNT
• Dual touch– The Pinch etc.
• Multi touch– possible contradictions
• Shape-based– Bringing physics to the surface
• Tangible UIs on surfaces– URP, illuminating light, SLAP, eLabBench
31
![Page 32: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/32.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Bumptop - the original video
32
![Page 33: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/33.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Making fun of Bumptop - discussion
33
![Page 34: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/34.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
RNT - Rotate and Translate with 1 Touch Point
• Problem: only 1 touch point (= mouse)• Goal: rotate and translate in a single gesture• Idea: use a physical model (inertia, friction)
– friction force is opposite to the direction of movement– rotation is recalculated in every frame
34
![Page 35: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/35.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Rotate, Translate and Scale with 2 Touch Points• track 2 points from frame to frame• compute scaling from change in distance d to d‘• compute rotation from angle between <d,d‘>• compute translation t and use directly
35
d
d‘
d
t
![Page 36: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/36.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Possible Contradictions with >2 Touch Points
• Consider R+T+S method for 2 TP• With 3 TP we can find 3 pairs• they will almost certainly yield
– different d, d‘– different orientation– possibly even different translations
• How to deal with this?– ignore 1 point ;-) looks weird in certain cases– use mean R,T,S to minimize error– deform the underlying object
36
d d‘
d
t
![Page 37: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/37.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Shape-based interaction
37
• Interaction in the real world uses not just contact points
• We use whole hands, arms, tools• Cannot be adequately expressed
using just contact points• How can we deal with this?
• Remember the lava lamp in Jeff Han‘s TED talk?
• Seriously: How can we do useful stuff with this?
![Page 38: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/38.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Tangible UIs on Interactive Surfaces• classic TUI project: URP• another classic: illuminating light• SLAP widgets
• Other examples already seen (hence not repeated): – MetaDesk– PhotoHelix
38
![Page 39: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/39.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 39
Luminous room: Urban Planning (URP)(John Underkoffler and Hiroshi Ishii, CHI 99)
• Move physical models of houses on a desk surface
• Simulate in the computer:– Shadows– Window reflections– Air flow and wind
![Page 40: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/40.jpg)
40
![Page 41: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/41.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 41
Luminous room: Illuminating Light (John Underkoffler and Hiroshi Ishii, CHI 98)
• Simulation of optical/holographic setups• Phys. objects represent optical elements• Top projection of resulting laser beam
![Page 42: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/42.jpg)
42
![Page 43: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/43.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 43
SLAP Widgets [Weiss et al. CHI 2009]
• Transparent Tangible objects for interactive surfaces• Use optical sensing (FTIR + DI)• Can be augmented with digital content• Idea: flexible multi-purpose tangible widget set
• SLAP Widget set:– a) Keypads. – b) Knob. – c) Slider. – d) Keyboard.
![Page 44: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/44.jpg)
44
![Page 45: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/45.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 45
eLabBench [Tabard et al., ITS 2011 best paper]
• Idea: make a Biology lab bench interactive• Use the same lab instruments as before• Augment them with digital information
![Page 46: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/46.jpg)
46
![Page 47: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/47.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Multi Display Interaction• Historic: Augmented Surfaces, Pick&drop• newer: bump, stitch• recent example: Touch projector
47
![Page 48: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/48.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Augmented SurfacesJun Rekimoto and Masanori Saitoh, CHI'99
• Combination of mobile devices and projection surfaces
• Interaction techniques:–hyperdragging–pick-and-drop–pick-and-beam–digital attachment
• interaction objects for tangible interaction• Camera-based acquisition of images• Selection from physical catalogues
Video
48
![Page 49: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/49.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide 49
![Page 50: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/50.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
TouchProjector [Boring et al., CHI `11]
50
![Page 51: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/51.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Stitching & Bumping [Hinckley et al.]
51
![Page 52: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/52.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Environment
context and task
challenges
input technologies
challenges in interaction design
output technologies
Take-away Message• many visual output technologies
• displays• tabletops• (steerable) projection
• instrumented environment = many displays• think about coordination• interaction techniques need to be invented
• No clear interaction concept yet• every device works differently• think about rules that can be applied to all• field of ongoing research!!
52
![Page 53: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/53.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Additional Literature• Rotation and translation mechanisms for tabletop interaction.
Mark S. Hancock, Frédéric Vernier, Daniel Wigdor and Sheelagh Carpendale, and Chia Shen. In Proc. Tabletop, pp. 79-86, 2006.– (also the source of the image on slide 25)
• A. D. Wilson, S. Izadi, O. Hilliges, A. Garcia-Mendoza, D. Kirk: „Bringing Physics to the Surface“, ACM UIST 2008 - Best Paper Award
• Malte Weiss, Julie Wagner, Yvonne Jansen, Roger Jennings, Ramsin Khoshabeh, James D. Hollan, and Jan Borchers. SLAP Widgets: Bridging the Gap Between Virtual and Physical Controls on Tabletops. In CHI '09:
• A. Tabard, J.D. Hincapié-Ramos, M. Esbensen, and J.E. Bardram: „The eLabBench: An Interactive Tabletop System for the Biology Laboratory“. ACM ITS 2011 - Best Paper Award
53
![Page 54: Mensch-Maschine-Interaktion 2 Interactive Environments · FogScreen • A wall sized, immaterial projection display • Image projected on dry, non-hazardous fog (pure water) •](https://reader034.fdocuments.us/reader034/viewer/2022042810/5f9e723a6b71671f0c478ce9/html5/thumbnails/54.jpg)
LMU München — Medieninformatik — Andreas Butz — !Mensch-Maschine-Interaktion II — WS2013/14 Slide
Environment
context and task
challenges
input technologies
challenges in interaction design
output technologies
Michel Beaudouin-Lafon:Instrumental Interaction in Multisurface
Environments
54
February 5th, 201410:15 - 11:45here in this room