Interaction modalities, technologies and tools for interactive art
-
Upload
jorge-cardoso -
Category
Education
-
view
3.456 -
download
3
description
Transcript of Interaction modalities, technologies and tools for interactive art
![Page 1: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/1.jpg)
Interaction modalities, technologies, and tools for
interactive art Jorge C. S. Cardoso
http://jorgecardoso.eu
![Page 2: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/2.jpg)
Objectives
• Overview of most common non-desktop interaction and computing paradigms
• Examples of technologies and tools that can be used to implement those interactions o Available tools at EArtes/UCP
• Most prominent people/references in these areas
• No code (for now)
![Page 3: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/3.jpg)
Topics Gestural interaction Tangible interaction Brain interaction
Wearable computing Physical computing Digital public displays Mobile computing
![Page 4: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/4.jpg)
Gestural interaction
Photo credits: Flickr user artmakesmesmile: http://www.flickr.com/photos/artmakesmesmile/169193698/
![Page 5: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/5.jpg)
Gestural interaction
"A gesture is a motion of the body that contains information. Waving goodbye is a gesture. Pressing a key on a keyboard is not a gesture because the motion of a finger on its way to hitting a key is neither observed nor significant. All that matters is which key was pressed".
-- Kurtenbach and Hulteen (1990), cited by Billinghurst & Buxton (2011)
![Page 6: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/6.jpg)
Gestural interaction
Technologies/Tools (ordered by the complexity of gestures: simplest to more complex) • Touch surfaces • Computer vision techniques • Wiimote • Kinect • MoCap
![Page 7: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/7.jpg)
Gestural interaction: Touch surfaces
• Mostly planar, rectangular, glass(y) surfaces • Various technologies for detecting touch
o which have different limits for the number of simultaneous touches it can detect
• Gestures o Fingers only (small devices) o Hands/forearms (large(r) devices, depending on technology
used) o 2d movements only
• Number of users o one - personal devices (phones, tablets) o few - large surfaces (tables, walls)
§ depends on the orientation of the surface
![Page 8: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/8.jpg)
Gestural interaction: Touch surfaces
Tablets/phones
• Singing fingers o Jay Silver o https://vimeo.com/12010952
Single-person interaction, fingers
![Page 9: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/9.jpg)
Gestural interaction: Touch surfaces
Medium/Large surfaces
• Touch-Wall @ EArtes o Laser light plane (LLP) technique
(see Sandler 2012) o Multi-touch surface (in
development) o Vertical orientation o One-two person interaction o Fingers, hands
![Page 10: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/10.jpg)
Gestural interaction: Touch surfaces
Medium/Large surfaces
• Bookstore @ EArtes
![Page 11: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/11.jpg)
Gestural interaction: Touch surfaces
Medium/Large surfaces
• Interactive tables (Reactables) o Diffused illumination technique o Horizontal orientation o Up to four people interacting o Fingers, hands shapes o Three available @ EArtes
![Page 12: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/12.jpg)
Gestural interaction: Touch surfaces
Very large surfaces
• Massive Curved Multi-Touch Screen o Reality touchscreen University of Groningen o http://www.youtube.com/watch?v=AlWFtF06RFo
Several-person interaction, fingers, hands
![Page 13: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/13.jpg)
Gestural interaction: computer vision
Computer vision (CV) techniques (non depth-cameras)
• Various kinds of surfaces/scenes
• Gestures o Varied, depends on positioning of camera
§ fingers § hands
§ whole body
o mostly 2d, but can also infer distance o not very precise
§ Can be very precise with controlled conditions
![Page 14: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/14.jpg)
Gestural interaction: computer vision
Video Place
• Myron Krueger
• http://www.youtube.com/watch?v=dmmxVA5xhuo
![Page 15: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/15.jpg)
Gestural interaction: computer vision
Edge/Swap
• Tiago Dionisio, Rudolfo Quintas
• http://www.youtube.com/watch?v=OSKpXZjECtY • http://www.youtube.com/watch?v=ryX4Vq2-IKE
![Page 16: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/16.jpg)
Gestural interaction: computer vision
BES Interactive Video Wall
• YDreams • http://www.youtube.com/watch?v=ijrZQbSOONQ
![Page 17: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/17.jpg)
Gestural interaction: computer vision
Balance
• Jorge Coutinho • https://vimeo.com/40506946
![Page 18: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/18.jpg)
Gestural interaction: computer vision
Sketchsynth
• Billy Keyes • http://vimeo.com/42053193 • http://golancourses.net/2012spring/05/13/billy-keyes-
final-project-sketchsynth/
![Page 19: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/19.jpg)
Gestural interaction: computer vision
All eyes on you
• Britzpetermann • https://vimeo.com/33186969
![Page 20: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/20.jpg)
Gestural interaction: computer vision
Knee deep
• Emily Gobeille & Theodore Watson • https://vimeo.com/8805152
![Page 21: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/21.jpg)
Gestural interaction: computer vision
Night lights
• YesYesNo • http://vimeo.com/8525186
![Page 22: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/22.jpg)
Gestural interaction: computer vision
aB
• Jorge Coutinho • http://vimeo.com/29524291
![Page 23: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/23.jpg)
Gestural interaction: Wiimote
Wiimote • Nintendo game controller
o Wireless: uses bluetooth to connect to a computer
o Has various accelerometer sensors to sense movement
o IR sensor for pointing
• Gestures o 3d movement, rotation o combined with the sensor bar, allows
pointing gestures
• Various wiimotes available @ EArtes
![Page 24: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/24.jpg)
Gestural interaction: Wiimote
Wiimote
• Wiimote hacks o Johnny Chung Lee o http://www.ted.com/talks/
johnny_lee_demos_wii_remote_hacks.html
![Page 25: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/25.jpg)
Gestural interaction: Wiimote
Pizza breakout
• Bruno Santos
![Page 26: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/26.jpg)
Gestural interaction: Wiimote
Wii Drum High Tutorial
• He Zhao • http://vimeo.com/2406224
![Page 27: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/27.jpg)
Kinect • XBox game controller
o Depth camera: senses the distance of each pixel to the camera (Borenstein 2012).
• Gestures o Detects a "point cloud" - pixels and their
distances to the camera o With extra software it's possible to
detect limbs (skeleton) positioning o More accurate than simple cameras
• Two available @ EArtes
Gestural interaction: Kinect
![Page 28: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/28.jpg)
Gestural interaction: Kinect
Kinect Physics Tutorial for Processing
• Video showing the result of Processing code that uses Kinect
• http://www.creativeapplications.net/processing/kinect-physics-tutorial-for-processing/
![Page 29: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/29.jpg)
Gestural interaction: Kinect
YScope medical interface
• YDreams • Gestural interface for an operating room • http://www.youtube.com/watch?v=91F6zErnCrs
![Page 30: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/30.jpg)
Gestural interaction: Kinect
Control the Humanoid Robot by Kinect
• Using the skeleton detected by the Kinect to drive a robot
• ikaziso • http://www.youtube.com/watch?v=w8BmgtMKFbY
![Page 31: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/31.jpg)
Gestural interaction: Kinect
Unnamed soundsculpture
• Daniel Franke & Cedric Kiefer • Using various Kinect to model a dancer and then
process the data into visual effects
• https://vimeo.com/38505448
• honorary mention from the Prix Ars Electronica
![Page 32: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/32.jpg)
Gestural interaction: Kinect
Make the line dance
• "1024" • Detection of skeleton + projection into human
body • https://vimeo.com/21308228
![Page 33: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/33.jpg)
Gestural interaction: Kinect
Fragmentos de um olhar
• Pedro Serrano
![Page 34: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/34.jpg)
Gestural interaction: LeapMotion
Leap Motion
• New controller for hand interactions • https://leapmotion.com/
![Page 35: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/35.jpg)
Gestural interaction: MoCap
MoCap - Motion Capture
• MoCap Room
• High-speed infrared cameras detect markers placed on actor's body
• Gestures o Highly accurate gestures
• (Vicon MoCap available @ EArtes)
![Page 36: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/36.jpg)
Gestural interaction: MoCap
T(ether)
• Tangible Media Group • Interaction with virtual volumetric
data • http://vimeo.com/42173010
![Page 37: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/37.jpg)
Gestural interaction: MoCap
Portrait of the ghost drummer
• Odaibe • Visualization of the process of
playing on a drum kit • http://vimeo.com/34682556 • Non-interactive
![Page 38: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/38.jpg)
Gestural interaction: computer vision
Computer Vision Tools
• Processing o BlobDetection o JMyron
• Max/Msp o Jitter o jit.cv external
• Community Core Vision o http://ccv.nuigroup.com/
![Page 39: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/39.jpg)
Gestural interaction: Wiimote
Wiimote Tools
• Processing/Java o Motej - http://motej.sourceforge.net/ o Wigee - http://www.wiigee.org/
• Max/Msp o aka.wiiremote external -
http://www.iamas.ac.jp/~aka/max/
![Page 40: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/40.jpg)
Gestural interaction: Kinect
Kinect Tools
• http://openkinect.org/wiki/Main_Page • Book "Making things see" (Borenstein 2012) • Processing/Java
o OpenKinect - http://www.shiffman.net/p5/kinect/ o SimpleOpenNI -
http://code.google.com/p/simple-openni/ o http://www.creativeapplications.net/processing/
kinect-physics-tutorial-for-processing/ • Max/Msp
o jit.freenect.grab - http://jmpelletier.com/freenect/
![Page 41: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/41.jpg)
Gestural interaction: Touch surfaces
Tools
• Web apps o jQMultiTouch library
§ http://dev.globis.ethz.ch/jqmultitouch/
![Page 42: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/42.jpg)
Gestural interaction
References
• Billinghurst & Buxton (2011). Gesture Based Interaction, http://www.billbuxton.com/input14.Gesture.pdf
• Borenstein, Greg (2012). Making Things See: 3D vision with Kinect, Processing, Arduino, and MakerBot, O'Reilly Media / Make
• Buxton, Bill (2012). Multi-Touch Systems that I Have Known and Loved, http://www.billbuxton.com/multitouchOverview.html
• Kurtenbach, G. & Hulteen, E. (1990). Gestures in Human-Computer Communications. In B. Laurel (Ed.) The Art of Human Computer Interface Design. Addison-Wesley, 309-317, http://www.billbuxton.com/inputManuscript.html
• Levin, Golan. Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers, http://www.flong.com/texts/essays/essay_cvad/
• Sandler, Seth (2012). LASER LIGHT PLANE MULTITOUCH TECHNIQUE, http://sethsandler.com/multitouch/llp/
![Page 43: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/43.jpg)
Tangible interaction
Photo credits: Flickr user kedume: http://www.flickr.com/photos/kedume/746632510/sizes/l/in/photostream/
![Page 44: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/44.jpg)
Tangible interaction
"[...] tangible interfaces give physical form to digital information, employing physical artifacts both as representations and controls for computational media. TUIs couple physical representations (e.g., spatially manipulable physical objects) with digital representations (e.g., graphics and audio), yielding user interfaces that are computationally mediated, but generally not identifiable as "computers" per se." - Ullmer & Ishii (2000)
![Page 45: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/45.jpg)
Tangible interaction
Picture from Ullmer and Ishii (2000).
![Page 46: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/46.jpg)
Tangible interaction
Tangible Media Group site
• http://tangible.media.mit.edu/projects/
![Page 47: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/47.jpg)
Tangible interaction
Reactable
• Tangible musical instrument o Users manipulate physical objects to create and control the sound
o Can be used as a general tangible framework
• http://www.youtube.com/watch?v=Mgy1S8qymx0
![Page 48: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/48.jpg)
Tangible interaction
Little Boxes
• ECAL/Joelle Aeschlimann • Music boxes for iPad • https://vimeo.com/45704273
![Page 49: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/49.jpg)
Tangible interaction
Sketch-a-TUI
• Alexander Wiethoff, et al. • Set of printable objects for tangible prototypes • http://vimeo.com/38793875 • Paper
o http://tei-conf.org/12/Main/Sketchatui
• Printable objects o http://project-premium.org/sketch-a-tui/index.html
![Page 50: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/50.jpg)
Tangible interaction
Birds on Paper • Chen-Wei Chiang, et al. • Drawing music • http://www.youtube.com/watch?v=qqkkNotSQMo • Paper
o http://tei-conf.org/12/Main/Birdsonpaper
![Page 51: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/51.jpg)
Tangible interaction
Chronotape • Peter Bennett • Tangible timeline with annotations • https://vimeo.com/28191512 • Paper
o http://tei-conf.org/12/Main/Chronotape
![Page 52: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/52.jpg)
Tangible interaction
Splash Controllers
• Luc Geurts and Vero Vanden Abeele • Uncareful Manipulation of Water • Paper
o http://tei-conf.org/12/Main/Splashcontrollers
![Page 53: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/53.jpg)
Tangible interaction
Skube
• Andrew Nip, Ruben van de Vleuten, Malthe Borch, and Andrew Spitz
• Tangible interface to Last.fm & Spotify Radio • http://www.creativeapplications.net/maxmsp/skube-
tangible-interface-to-last-fm-spotify-radio/
![Page 54: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/54.jpg)
Tangible interaction
Faustine
• Carlos Caires & Jorge Cardoso • Interactive video installation inspired by the short story
"Morel's Invention" • https://vimeo.com/19043202
![Page 55: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/55.jpg)
Tangible interaction
The garden of time
• Carlos Caires & Jorge Cardoso • Interactive video installation inspired by the short story
"The Garden of Forking Paths" • https://vimeo.com/24836769
![Page 56: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/56.jpg)
Tangible interaction
Tools
• Reactable • TUIO - Fiducial tracking
o http://www.tuio.org/?software o Processing o Max/MSP o CCV
• Wiimotes • Other off-the-shelf hardware/sensors
![Page 57: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/57.jpg)
Tangible interaction
References • G. W. Fitzmaurice, H. Ishii, and W. A. S. Buxton, “Bricks: Laying the
Foundations for Graspable User Interfaces,” in Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’95, 1995, pp. 442–449.
• H. Ishii and B. Ullmer, “Tangible bits,” in Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’97, 1997, pp. 234–241.
• Ullmer, B. and Ishii, H. (2000). “Emerging frameworks for tangible user interfaces,” IBM Systems Journal, vol. 39, no. 3, pp. 915–931, Jul. 2000.
![Page 58: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/58.jpg)
Brain interfaces
Photo credits: Flickr user Saad Faruque: http://www.flickr.com/photos/cblue98/7254347346/sizes/l/in/photostream/
![Page 59: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/59.jpg)
Brain interfaces
Brain-Computer Interfaces (BCI) or Mind-Machine Interface (MMI)
Using brain signals to interact with computers.
• Usually based on EEG readings • Various commercial devices available • http://en.wikipedia.org/wiki/Brain
%E2%80%93computer_interface#Low-cost_BCI-based_Interfaces
![Page 60: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/60.jpg)
Brain interfaces
Câmara Neuronal
• Adolfo Luxúria Canibal, João Martinho Moura, Miguel Pedro Guimarães
• http://projects.jmartinho.net/3486412/Camara-Neuronal-Video-Teaser
![Page 61: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/61.jpg)
Brain interfaces
Brainwave beats
• Collin Cunningham • Midi notes from brain signals • Uses the Neurosky Mindset device • http://blog.makezine.com/2011/05/05/collins-lab-
brainwave-beats/
![Page 62: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/62.jpg)
Brain interfaces
Mansion Maze 3 BCI Game Trailer
• Eric Rosendale, Ian McCabe, Veronica Cole, Aaron DeChamplain, and Matt Stephan
• BCI Game • http://www.youtube.com/watch?v=sdYAIrr9nDw
![Page 63: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/63.jpg)
Brain interfaces
Mind Control of Lego NXT Telepresence Robot
• Mark Bruce • Controling a lego robot using brain signals • Uses EPOC BCI device • https://www.youtube.com/watch?v=nX741DZw8l4
![Page 64: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/64.jpg)
Brain interfaces
Tools
• Neurosky Mindset (available @ EArtes) • EPOC neuroheadset - http://emotiv.com/emortal/cart/ • Processing library
o Neurosky Mindset Processing § http://jorgecardoso.eu/processing/
MindSetProcessing/ • http://en.wikipedia.org/wiki/Brain
%E2%80%93computer_interface#Low-cost_BCI-based_Interfaces
![Page 65: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/65.jpg)
Wearable computing
Geordi La Forge, Star Trek The Next Generation character wearing the VISOR
![Page 66: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/66.jpg)
Wearable computing
"Wearable computing is the study or practice of inventing, designing, building, or using miniature body-borne computational and sensory devices. Wearable computers may be worn under, over, or in clothing, or may also be themselves clothes" (Mann, 2012)
![Page 67: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/67.jpg)
Wearable computing
Image from: http://www.eyetap.org/research/wearables/wearcomp/wearables.html
Image from: http://www.interaction-design.org/
![Page 68: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/68.jpg)
Wearable computing
Google Project Glass
• https://plus.google.com/+projectglass/posts • https://www.youtube.com/watch?
v=9c6W4CCU9M4
(how will our world be in 5/10 years, when we all start wearing devices even more powerful than Glass?)
![Page 69: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/69.jpg)
Wearable computing
XS Labs
• http://xslabs.net/
![Page 70: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/70.jpg)
Wearable computing
Blazer 01.2
• A light emitting bracelet • Can be used to "draw" letters in the air
• Vincent leclerc • http://www.uttermatter.com/blazer/012/index.htm#
![Page 71: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/71.jpg)
Wearable computing
cidZiy
• Capacitive sensing textile surface
• xslabs • http://www.youtube.com/watch?v=zH4J6Uaf88A
![Page 72: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/72.jpg)
Wearable computing
Kukkia
• Kukkia flowers frame the face and slowly open and close over time
• xslabs • http://www.youtube.com/watch?v=R5fd6AwGMhg
![Page 73: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/73.jpg)
Wearable computing
See U in my Tee
• Inês Petiz • T-shirts that react to the presence of other t-shirts • http://wearable-technology.blogspot.pt/
![Page 74: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/74.jpg)
Wearable computing
References • Mann, Steve (2012): Wearable Computing. In: Soegaard, Mads and Dam,
Rikke Friis (eds.). "Encyclopedia of Human-Computer Interaction". Aarhus, Denmark: The Interaction Design Foundation. Available online at http://www.interaction-design.org/encyclopedia/wearable_computing.html
![Page 75: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/75.jpg)
Physical computing
![Page 76: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/76.jpg)
Physical computing
"Physical computing, in the broadest sense, means building interactive physical systems by the use of software and hardware that can sense and respond to the analog world. (...) In the broad sense, physical computing is a creative framework for understanding human beings' relationship to the digital world. In practical use, the term most often describes handmade art, design or DIY hobby projects that use sensors and microcontrollers to translate analog input to a software system, and/or control electro-mechanical devices such as motors, servos, lighting or other hardware." - (Physical computing 2012)
![Page 77: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/77.jpg)
Physical computing
Physical computing book
• O'Sullivan & Igoe 2004
![Page 78: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/78.jpg)
Physical computing
reaDIYmate
• Olivier Mével and Marc Chareyron
• http://www.readiymate.com
![Page 79: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/79.jpg)
Physical computing
An Instrument for the Sonification of Everyday Things
• Dennis P Paul • http://
www.creativeapplications.net/processing/an-instrument-for-the-sonification-of-everyday-things/
![Page 80: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/80.jpg)
Physical computing
Piano playing stairs
• http://www.youtube.com/watch?v=DuNWkKBokts
![Page 81: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/81.jpg)
Physical computing
What the phonics
• Andrew Spitz & Momo Miyazaki • https://vimeo.com/45747333
![Page 82: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/82.jpg)
Physical computing
Barcode band
• Kang woon jin, Lee ha lim, Kim yong duk • https://vimeo.com/41132461
![Page 83: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/83.jpg)
Physical computing
Syndyn
• André Rangel and Anne-Kathrin Siegel • https://vimeo.com/20859799
![Page 84: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/84.jpg)
Physical computing
Little Bits
• Hardware components that can be assembled together • http://market.littlebits.cc/ • https://vimeo.com/45276780
![Page 85: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/85.jpg)
Physical computing
MakeyMakey
• Circuit board that connects to a computer via USB and allows everyday objects to become controllers
• http://www.makeymakey.com/
![Page 86: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/86.jpg)
Physical computing
Lego Mindstorms NXT
• Available @ EArtes • Lego kit with
o motors o distance sensor o light sensor o button o lego parts...
![Page 87: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/87.jpg)
Physical computing
Tools • Arduino - http://www.arduino.cc/ • Galago - http://outbreak.co/galago • Digispark - http://digistump.com/ • Raspberry pi - http://www.raspberrypi.org/ • MakeyMakey -
http://www.makeymakey.com/ • LittleBits - http://littlebits.cc/ • Lego NXT Mindstorms (available @EArtes)
o http://mindstorms.lego.com/en-us/Default.aspx
• 3d printer, CNC, etc.
![Page 88: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/88.jpg)
Physical computing
References • Physical computing. (2012, July 21). In Wikipedia, The
Free Encyclopedia. Retrieved 14:37, September 22, 2012, from http://en.wikipedia.org/w/index.php?title=Physical_computing&oldid=503476286
• Dan O'Sullivan, Tom Igoe. (2004). Physical Computing: Sensing and Controlling the Physical World with Computers, Thomson Course Technology, Boston, Massachusetts,
![Page 89: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/89.jpg)
Digital public displays
Photo credits: Flickr user bmann: http://www.flickr.com/photos/boris/5104900447/sizes/l/in/photostream/
![Page 90: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/90.jpg)
Digital public displays
Digital displays located in public areas, allowing anyone to see/interact with them
• Traditionally, public displays have been used to broad information from one active source to millions of passive receivers
• Digital public displays can change this o Turning receivers into active receivers o Supporting dynamic locally relevant information o Supporting user-generated content (many to many
content distribution) • Everyone can interact
o Everyone can see you interacting o Everyone sees the results from the interaction
![Page 91: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/91.jpg)
Digital public displays
Advertising
![Page 92: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/92.jpg)
Digital public displays
Honda Interactive Billboard using SMS and Bluetooth
• Users can "start" the car via SMS • http://www.youtube.com/watch?v=OudaJGtwdQ8
![Page 93: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/93.jpg)
Digital public displays
BBC Billboard
• Voting billboards, users can SMS their vote • http://www.bbc.co.uk/blogs/theeditors/2007/06/
live_in_new_york_city.html
![Page 94: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/94.jpg)
Digital public displays
Audio Jack Billboards
• Promoting a TV show "Big Love" • Users can listen to people's thoughts by plugin
headphones into the billboard • http://www.trendhunter.com/trends/audio-jack-billboard-
for-big-love
![Page 95: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/95.jpg)
Digital public displays
Nokia's N90 interactive billboard
• YDreams • The billboard snaps people's pictures and displays them
on the billboard • http://www.youtube.com/watch?v=UWgIswo3lV4
![Page 96: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/96.jpg)
Digital public displays
Mini Billboard Interactive
• The billboard displays the names of the Mini car owners passing by
• http://www.nytimes.com/2007/01/29/business/media/29cooper.html
![Page 97: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/97.jpg)
Digital public displays
Nikon D700
• The billboard plays flashing cameras as people pass by. • http://www.thecoolhunter.net/article/detail/1570/nikon-
d700-guerrilla-style-billboard
![Page 98: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/98.jpg)
Digital public displays
Ambient displays
• Information about their surroundings • Information visualization, aesthetics, art
![Page 99: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/99.jpg)
Digital public displays
Hello Wall
• (Prante et al. 2003) • Can display various abstract information patterns • Can display group activity
![Page 100: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/100.jpg)
Digital public displays
Informative Art ambient display
• Skog et al. 2003 • Display bus information (only the bus number 16 - two in
each direction) o Size of square: amount of time before bus leaves o Color: Time to leave to the bus stop o Position: right - buses to the city center; left - buses from
the city center
![Page 101: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/101.jpg)
Digital public displays
Information Percolator
• Heiner et al. 1999 • A water bubble display
o Can be used to display various types of ambient information
• http://www.cs.cmu.edu/~hudson/video.html
![Page 102: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/102.jpg)
Digital public displays
Entertainment/Waiting times
![Page 103: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/103.jpg)
Digital public displays
Mobilenin
• Scheible, Jürgen and Ojala, Timo • Paper
o http://www.mendeley.com/research/mobilenin-combining-multi-track-music-video-personal-mobile-phones-public-display-multi-user-interac
o Allows users to vote on the next video clip
![Page 104: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/104.jpg)
Digital public displays
Public Youtube Player
• Allows users to have some control over the videos that are played
• at the EArtes' Bar display
![Page 105: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/105.jpg)
Digital public displays
Everybody Votes
• Users can express their opinions regarding various questions and suggest their own questions
• at the EArtes' Bar display
![Page 106: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/106.jpg)
Digital public displays
References
Heiner, J. M., Hudson, S. E., & Tanaka, K. (1999). The Information Percolator : Ambient Information Display in a Decorative Object. Proceedings of the 12th annual ACM symposium on User interface software and technology - UIST ’99 (pp. 141–148). New York, New York, USA: ACM Press. doi:10.1145/320719.322595
Prante, T., Röcker, C., Streitz, N., Stenzel, R., Magerkurth, C., van Alphen, D., & Plewe, D. (2003). Hello.Wall - Beyond Ambient Displays. Video Track and Adjunct Proceedings of the 5th Intern. Conference on Ubiquitous Computing (UBICOMP’03). Seattle, Wash., USA.
Skog, T., Ljungblad, S., & Holmquist, L. E. (n.d.). Between aesthetics and utility: designing ambient information visualizations. IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714) (pp. 233–240). IEEE. doi:10.1109/INFVIS.2003.1249031
![Page 107: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/107.jpg)
Mobile computing
![Page 108: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/108.jpg)
Mobile computing
"Mobile computing is human–computer interaction by which a computer is expected to be transported during normal usage. Mobile computing involves mobile communication, mobile hardware, and mobile software." - (Mobile computing 2012)
![Page 109: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/109.jpg)
Mobile computing
X-Fetch
• Daniel Santos, João Rema • City-wide game, peddy paper like • Players have to discover a set of clues spread across
the city (Porto) using a mobile device that plays videos pointing to next clues and that tracks players GPS locations
• http://jpn.c2com.up.pt/2005/10/29/xfetch_quando_o_real_e_o_virtual_se_confundem.html
![Page 110: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/110.jpg)
Mobile computing
• GPS Drawing
Jeremy Wood http://www.gpsdrawing.com/gallery.html
![Page 111: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/111.jpg)
Mobile computing
Mobile mobile
• http://www.theophane.co.uk/mobile-mobile • https://vimeo.com/8118881
![Page 112: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/112.jpg)
Mobile computing
Pom
• Jorge Coutinho • Jogo Pong estendido a seis jogadores • https://vimeo.com/18579379
![Page 113: Interaction modalities, technologies and tools for interactive art](https://reader033.fdocuments.us/reader033/viewer/2022052822/554e8da6b4c905fc368b4a8a/html5/thumbnails/113.jpg)
Mobile computing
References
• Mobile computing. (2012, September 8). In Wikipedia, The Free Encyclopedia. Retrieved 17:56, September 30, 2012, from http://en.wikipedia.org/w/index.php?title=Mobile_computing&oldid=511364202