Compliance storyboard: “Classifying & controlling content at the input device”
Classifying 3D Input Devices - WPI
Transcript of Classifying 3D Input Devices - WPI
![Page 1: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/1.jpg)
Robert W. Lindeman Associate Professor
Interactive Media & Game Development Department of Computer Science Worcester Polytechnic Institute
IMGD 5100: Immersive HCI
Classifying 3D Input Devices
![Page 2: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/2.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
2
Motivation The mouse and keyboard are good for general desktop
UI tasks Text entry, selection, drag and drop, scrolling, rubber
banding, … Fixed computing environment 2D mouse for 2D windows
How can we design effective techniques for 3D? Use a 2D device? Use multiple n-D devices? Use new devices? Use 2D interface widgets? Need new interaction techniques!
![Page 3: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/3.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
3
Motivation (cont.) Gaming and Virtual Reality
Tight coupling between action and reaction Need for precision
VR can give real first-person experiences, not just views Head-mounted Display
In order to look behind you, turn your head! Selecting/manipulating an object
Reach your hand out and grab it! Travel
Just walk (well, not quite)!
Doing things that have no physical analog is more problematic
![Page 4: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/4.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
4
Common Input Devices
TrackPoint TrackBall TrackPad
Mouse Keyboard Joystick
Tablet MightyMouse Multi-Touch
TrackPad
![Page 5: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/5.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
5
Game Controllers
Atari 2600 (1977)
Intellivision (1980)
PlayStation2 (2000)
Xbox 360 (2005)
PlayStation3 (2008)
![Page 6: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/6.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
6
"Natural" Motion Controllers
Microsoft Kinect (2010?)
PlayStation Move (2010)
WiiMote (2007)
WiiMotionPlus (2009)
Razor Hydra (2013)
Leap Motion (2013)
![Page 7: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/7.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
7
Multi-Touch Surfaces High resolution Co-located interaction
http://www.ted.com/talks/jeff_han_demos_his_breakthrough_touchscreen.html
CLIP
![Page 8: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/8.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
8
Prototypes of Controllers
Nintendo “Revolution” Controller (prototype)
Nintendo Wii + Nunchuck (released)
CLIP
![Page 9: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/9.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
9
Prototypes of Controllers (cont.)
PlayStation3 Controller (prototype)
PlayStation3 SIXAXIS (released)
![Page 10: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/10.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
10
Hand-Held Devices Becoming interesting!
Apple iPhone 4 (2010)
Apple iPad (2010)
Motorola DROID (2009)
Sony PlayStation Portable (2004)
Nintendo DS Lite (2006)
![Page 11: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/11.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
11
Classification Schemes Relative vs. Absolute movement Integrated vs. Separable degrees of
freedom Digital vs. Analog devices Isometric vs. Isotonic devices Rate control vs. Position control Special-purpose vs. General-purpose
devices Direct vs. Indirect manipulation
![Page 12: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/12.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
12
More on Classifications Relative vs. Absolute movement
Mouse vs.Tablet
Integrated vs. Separable degrees of freedom Mouse has integrated X, Y control Etch-a-sketch has separate X, Y control
Motions that are easy with one are hard with the other
Analog devices allow more sensitivity For example, analog game controllers
![Page 13: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/13.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
13
Isometric vs. Isotonic Input Devices (Zhai) No motion vs. No resistance Actually a continuum of elasticity
TrackPoint (mostly isometric) vs. mouse (mostly isotonic)
Many devices are re-centering (e.g., joysticks)
![Page 14: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/14.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
14
Rate Control vs. Position Control (Zhai) Mouse is normally used for position control Mouse scroll-wheel
Position control Click-drag for rate controlled scrolling
Trackballs typically use position control Joysticks: Control position (cross-hair), or
Control velocity (aircraft) Rate control eliminates need for clutching/
ratcheting Isotonic-rate control and isometric-position
control tend to produce poor performance (Zhai)
![Page 15: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/15.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
15
Special-Purpose vs. General-Purpose Input Devices (Buxton) Game controllers are designed to support
many types of games Game developer decides on mapping No "standard" mappings -> each game
different
Some special-purpose devices exist Light guns Steering wheels RPG keyboard/joystick Drum kits, dance pads, bongos, etc.
![Page 16: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/16.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
16
Direct vs. Indirect Manipulation Direct
Clutch and drag an icon with mouse or stylus Touch screens, PDAs use direct manipulation Works well for things that have a physical analog
Indirect Use some widget to indirectly change something
Problems with direct manipulation Some things do not have a physical analog Precision may be lacking Selection/de-selection may be messy
![Page 17: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/17.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
17
PHANTOM Omni Haptic Device
3D Input Devices
SpaceBall SpaceMouse
HMD with 3-DOF tracker
Tracked Paddle for 2D Interaction
CyberGlove II
![Page 18: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/18.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
18
Motion-Capture/Tracking Systems Used heavily in movies and TV
Capture actual motion, and re-use Example, Fox Sports NFL guy
Can be done interactively, or offline Can capture three or more (six) Degrees
of Freedom (DoF) Position, Orientation, or Both
Many technical approaches No really good, general approaches
![Page 19: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/19.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
19
Tracking Technologies Mechanical Magnetic Ultrasonic Inertial Optical Time of flight Hybrid
![Page 20: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/20.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
20
Mechanical Tracking Rigid linkage, potentiometers at joints Pros:
High accuracy High resolution
Cons: Limited range of motion Cumbersome
![Page 21: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/21.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
21
Magnetic Tracking Transmitter creates a magnetic field
Transmitter is the origin
Receivers are tracked using changes in magnetic field
Pros: Fairly lightweight Six DoF
Cons: Very noisy near ferrous metal Limited working range
![Page 22: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/22.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
22
Ultrasonic Tracking Transmitter sends pulses Receivers hear tones Distance is computed Can use "costellations" for orienation Pros:
High accuracy High resolution
Cons: Requires line-of-sight (hearing)
![Page 23: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/23.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
23
Inertial Tracking Accelerometers
Tilt Acceleration
Gyroscopes Measure movement
Pros: Not anchored to a place in space
Cons: Accumulated error can cause drift Only moderate accuracy
![Page 24: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/24.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
24
Optical Tracking Multiple fixed cameras capture markers Known camera parameters (FOV, focal
length, position, orientation) Use equations to compute position in 3-D
space Markers can be simple points, or glyphs ARToolKit
http://sourceforge.net/projects/artoolkit/
![Page 25: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/25.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
25
Optical Tracking (cont.) Active vs. Passive Markers
![Page 26: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/26.jpg)
Kinect Structured light + sensor
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
26
![Page 27: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/27.jpg)
Kinect Star Wars
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
27
![Page 28: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/28.jpg)
Kinect Fusion
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
28
![Page 29: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/29.jpg)
Kinect IllumiRoom
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
29
![Page 30: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/30.jpg)
PlayStation MOVE Camera tracker + inertial tracker
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
30
![Page 31: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/31.jpg)
Leap Motion Three IR LEDs
Illuminate hands, etc.
Two stereo cameras Sends grey-scale
images to Leap Motion Service
Not depth map! Software tracks
fingers using hand-information
Temporal coherence R.W. Lindeman - WPI Dept. of Computer Science
Interactive Media & Game Development 31
![Page 32: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/32.jpg)
Leap Motion
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
32
![Page 33: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/33.jpg)
Leap Motion UX Design, Part 1
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
33
![Page 34: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/34.jpg)
Leap Motion UX Design, Part 2
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
34
![Page 35: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/35.jpg)
Leap Motion UX Design http://blog.leapmotion.com/category/ux/
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
35
![Page 36: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/36.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
36
Hybrid Tracking Techniques Compensate negative characteristics of
one approach with another Inertial and Magnetic Inertial and Optical WiiMote+MotionPlus PlayStation Move
![Page 37: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/37.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
37
Other Options Some alternatives
Speech Gestures: pointing to fly Device actions (e.g., buttons, joysticks) Head/gaze directed
Hybrid Speech and gesture (e.g., "Put that, there.")
![Page 38: Classifying 3D Input Devices - WPI](https://reader033.fdocuments.us/reader033/viewer/2022051801/628168ee441f3e6ab268d0e4/html5/thumbnails/38.jpg)
R.W. Lindeman - WPI Dept. of Computer Science Interactive Media & Game Development
38
Special-Purpose Input Devices Some applications are more "real" with a
device that matches the real action Steering wheel Light gun Flight-simulator motion platform Snowboard/surfboard Pod racer Motor cycle
Today, since sensors are cheap, we can turn almost anything into an input device